Microsoft has open sourced code-testing-generator, a polyglot agent that writes unit checks after which proves they work. It ships within the dotnet-test plugin contained in the MIT-licensed dotnet/expertise repository.
The agent targets a spot that coding assistants normally depart open. A immediate like ‘generate unit checks’ doesn’t say which framework, file location or assertions to make use of. code-testing-generator settles these selections by studying the repository earlier than it writes something. It then plans, writes, runs and checks the checks it produces. On Microsoft’s inner 152-task benchmark, it accomplished 140 duties towards 120 for inventory GitHub Copilot. Each setups used the identical mannequin and prompts.
Is it deployable
Sure. It’s an agent definition together with expertise, not a hosted service, so it runs inside your present coding agent and code stays native.
- Firm stage: viable from solo maintainers upward. Startups and mid-market groups acquire most, as a result of the agent provides repository analysis a small staff has no time to encode. Enterprises can fork the language steering to match inner frameworks.
- Industries: regulated or audit-heavy software program estates — monetary companies, healthcare, insurance coverage, public sector — plus platform groups paying down legacy take a look at debt.
- Purposes: backfilling checks on untested modules, producing checks for a pull-request diff, elevating protection earlier than a launch gate, and standardising conventions throughout polyglot monorepos.
What the agent really does
It coordinates work by a Analysis-Plan-Implement (RPI) pipeline. It searches the repository for code needing checks, detects the language and take a look at framework, reads present checks for conventions, and finds the true construct and take a look at instructions. That final step targets a selected failure: a take a look at mission that builds domestically however by no means runs in CI as a result of nothing registered it.
The agent then picks certainly one of three methods. Direct writes and validates checks instantly. Single move runs one cycle. Iterative repeats it for giant scopes or protection targets. It by no means modifies manufacturing code, and avoids checks that decision exterior URLs, bind ports or depend upon timing.
The verification gate
Earlier than reporting completion, the agent runs 5 checks. It causes about small code adjustments that ought to make the checks fail, a light-weight type of mutation testing. It seems to be for weak or lacking assertions. It maps each requested state of affairs to a take a look at. It builds the complete workspace and runs the complete suite. It confirms the repository’s personal take a look at command discovers the brand new checks.
Benchmark outcomes
On Microsoft’s inner benchmark of 152 duties from actual repositories, the agent accomplished 140 (92.1%) versus 120 (78.9%) for inventory GitHub Copilot on the identical mannequin and prompts (63% fewer failures).
The acquire is concentrated. On 89 imprecise prompts, the agent resolved 79 (88.8%) towards 59 (66.3%), slicing failures from 30 to 10. On 63 detailed prompts, each scored 61 (96.8%). On 15 duties focusing on a selected diff, the agent handed all 15 and inventory Copilot handed none.
Notably, the agent generated 2.3% fewer checks (6,963 vs 7,129) at successfully similar line protection (72.4% vs 72.2%). Common process time was 359 seconds towards 380. Token use per accomplished process was 3.2% greater.
On 45 .NET duties, Claude Opus 4.8 reached 43/45 with the agent versus 35/45 inventory; GPT-5.5 reached 41/45 versus 36/45. On the more durable exterior SWE Atlas benchmark, completion was 16/44 versus 12/44.
