The KwaiKAT Group at Kuaishou has launched the KAT-Coder-V2.5. It’s a coding mannequin educated to function inside actual, executable repositories moderately than emit single-turn code. The served mannequin is offered by means of StreamLake. An open-weight variant, KAT-Coder-V2.5-Dev, was launched individually on Hugging Face underneath Apache-2.0.
AutoBuilder: environments that really run the meant checks
The analysis frames a verifiable job as a triplet. It wants a exact job description, an executable repository setting, and a set of validation checks. A patch is appropriate provided that it passes all of them.
Duties are mined from actual pull requests and commits, following the SWE-bench lineage. The merged code change provides a golden patch and the accompanying take a look at change provides a take a look at patch. Uncooked problem textual content is discarded as a specification. As an alternative, descriptions are regenerated into three elements: an issue assertion grounded within the golden patch, necessities derived from the take a look at patch, and interface constraints inferred from each. A readability verify then drops something ambiguous, incomplete, underspecified, or internally inconsistent.
AutoBuilder handles the setting facet. A construct agent analyzes the repository and writes a configuration script that installs dependencies and runs checks from a clear checkout. A verification agent executes that script in an remoted sandbox.
The acceptance rule is the fascinating half. Verification doesn’t learn exit codes or grep log patterns. It parses structured test-framework output, and accepts an setting solely when greater than 90% of anticipated checks are collected and go/fail outcomes reproduce throughout runs. Failures are fed again as structured data for iterative restore.
Combining a preconfigured base setting, build-system templates, and a retrievable library of distilled construct recipes raised the development success fee from 16.5% to 57.2%. The result’s over 100,000 verifiable environments spanning 12 languages. Git historical past, commit metadata, and different exploitable traces are stripped so brokers can’t learn the reference resolution out of the repo.
Knowledge Scaling Flywheel: filtering on course of
Filtering trajectories by ultimate take a look at success is deceptive. Some passing runs depend on hard-coding, mechanism bypassing, or test-oriented shortcuts. Some failing runs include precious search, localization, and restore conduct.
KwaiKAT addresses each instructions. For close to misses, focused process-level hints point out what to examine or confirm with out revealing the answer. That alone raises the go fee of beforehand zero-pass duties to roughly 20%. As a result of hinted trajectories include data unavailable at inference, the verified patch is then mounted and a hint-free trajectory is regenerated from the unique job context. Solely samples that go verification, present no trace leakage, and keep in keeping with the patch are retained.
For passing runs, rule-based gates take away invalid, unstable, or exploitative trajectories. A scoring stage then charges exploration, localization, pre-edit reasoning, specification constancy, repository conventions, patch minimality, verification high quality, restoration conduct, and honesty.
A 3rd mechanism targets harness overfitting. Device names, argument conventions, output codecs, and immediate templates are randomized whereas performance is preserved. As a result of verification is anchored to check outcomes moderately than harness traces, one job may be re-served underneath many harness configurations. Practical perturbations are injected too: lacking dependencies, transient command failures, truncated outputs, and noisy logs.
Discover the pipeline and the numbers
Infrastructure failures capped rewards earlier than algorithmic limits
Throughout KAT-Coder-V2 coaching, gradual reward curves had been initially blamed on the RL algorithm. Nonetheless, an audit revealed that ~16% of trajectories failed as a consequence of sandbox infrastructure points moderately than the mannequin coverage, with boundary misalignments typically emptying observations for ~40 steps and corrupting rewards.
Three infrastructure fixes adopted. First, an early-release picture eviction coverage lowered disk utilization from 95% to 60%, decreasing timeout-induced invalid rollouts from 6–7% to underneath 1%. Second, correcting setting variables throughout distant sandbox initialization stopped system overrides that flipped rewards on 6–7% of samples, chopping errors under 1%. Third, the Gateway Server bypassed mainstream chat endpoints—which prompted 40% token drift at a ~200-turn scale by re-applying apply_chat_template and re-tokenizing—and known as /generate straight to make sure rollout token alignment.
Collectively, these updates decreased the sandbox suggestions error fee from roughly 16% to under 2% and minimize coaching collapses by an order of magnitude.
Uneven PPO and a three-tier reward
The analysis staff selected PPO with GAE over critic-free trajectory strategies as a result of manufacturing harnesses break up periods into structurally distinct samples, complicating group baselines.
Utilizing uneven actor–critic, the Critic will get privileged coaching context (rewards, checks, protection, patches, metadata, future turns), whereas the Actor sees solely rollout state. The Critic and further context are discarded at inference.
Rewards are three-tiered: Core Job Scores require all fail_to_pass and pass_to_pass checks to go; Commonplace Conduct Constraints penalize duplication, unhealthy software calls, and debug remnants; Failed Trajectory Incentives rating file retrieval by way of F2 and provides partial take a look at credit score.
5 consultants fuse by way of Multi-Instructor On-Coverage Distillation utilizing reverse KL, an off-policy begin, and drift-aware truncation from Prune-OPD.
Outcomes
Below a unified Claude Code harness, KAT-Coder-V2.5 leads its panel on PinchBench with 94.9, beating Opus 4.8 at 93.5. It locations second on SWE-Bench Professional (65.2 vs 69.2) and the interior KAT Code Bench (53.1 vs 57.3).
Nonetheless, it lags on Terminal-Bench 2.1, inserting final with 60.7 behind GLM-5.1 (61.8) and Opus 4.8 (84.6). On SciCode, it scores 50.3, matching GLM-5.2.
Notably, the open-weight KAT-Coder-V2.5-Dev is a separate 35B-total / 3B-active MoE post-trained on Qwen3.6-35B-A3B utilizing 127K SFT examples, then RL. Evaluated on a separate in-house protocol, its outcomes should not corresponding to the primary flagship desk.
Key Takeaways
- KwaiKAT treats agentic coding as an infrastructure drawback, not a model-scale drawback.
- AutoBuilder lifted setting building success from 16.5% to 57.2%, yielding 100,000+ verifiable environments throughout 12 languages.
- A sandbox audit discovered ~16% of RL trajectories failed due to the sandbox, not the coverage; fixes minimize that to under 2%.
- KAT-Coder-V2.5 tops PinchBench at 94.9 and ranks second on SWE-Bench Professional at 65.2, behind Opus 4.8.
- The open-weight KAT-Coder-V2.5-Dev is a separate 35B-A3B MoE underneath Apache-2.0, with its personal benchmark numbers.
Take a look at the Paper, the Mannequin Weight on Hugging Face, and the Product Web page. All credit score for this analysis goes to the researchers of this undertaking.
Michal Sutter is an information science skilled with a Grasp of Science in Knowledge Science from the College of Padova. With a stable basis in statistical evaluation, machine studying, and information engineering, Michal excels at reworking complicated datasets into actionable insights.
