Kozuchi Agent

01 · Overview

Small model, frontier results

Kozuchi Agent is a phase-driven software-repair agent that pairs a 27B open-weight model (Qwen3.5-27B) with reuse orchestration, K-candidate trajectory sampling, and an agent-internal selector — no closed-source verifier and no hidden-test access at selection time. It resolves 74.80% of SWE-bench Verified and transfers to Java (32.03% on Multi-SWE-bench) without language-specific changes.

SWE-bench Verified success rate vs. model parameters; Kozuchi at 74.8% with 27B sits on the Pareto front — Kozuchi mini-swe-agent + Qwen3.5-27B reaches 74.8% on SWE-bench Verified with ~10–40× fewer parameters than peers at comparable success rates. — click to open in `figures/`

02 · The Paper

Read the paper

arXiv:2608.15579 ↗

Click the snapshot to open the full PDF

Preprint on arXiv: arXiv:2608.15579 — accepted at ASE 2026 Industry Showcase (13 pages, 4 figures).
Six research questions covering headline performance, peer comparison, variance analysis, failure taxonomy, test-time-scaling selection lift, and cross-language transfer.
Statistically grounded: Wilson CIs, repo-level cluster bootstrap, McNemar paired tests with BH-FDR correction across 17 open-weight peers.
Fully rebuildable: paper/build.sh compiles the PDF offline with bundled ACM class files.
Every cited number maps to a script via INDEX.md; every reference is annotated in REFERENCES_INDEX.md.

📄 Camera-ready PDF 📚 arXiv PDF Abstract & BibTeX ↗

03 · Architecture

How it works

browse figures/ ↗

Industrial software-engineering teams increasingly prefer LLM agents that turn bug reports into correct, test-passing patches — but operating such agents at industrial scale raises harness-level problems: long horizons, tool-use discipline, context persistence, multi-cluster execution, and evaluation reuse. Kozuchi Agent answers with a language-agnostic open-weight agent and a CI-operated evaluation pipeline built on five pillars:

Explicit phasesAn 8-phase / 15-edge finite-state workflow (reproduce → test synthesis → localization → fix → verify → report) with 14 phase-gated skills.

Persistent intermediate stateDurable shared memory — scripts, generated tests, traces, handovers, patch notes — survives across phases and candidates.

Deterministic SE toolsThree active, deterministic software-engineering tools instead of free-form shell improvisation.

Model-independent actionsA turn-level action normalizer keeps the action interface identical across backbone models.

Cross-agent test-time selectionK=8 candidate trajectories per instance, ranked by a K×K selector using only agent-chosen tests — never hidden benchmark tests.

Auditable & repeatableNo proprietary models, no fine-tuning, no large backbones — behavior is reproducible from a locally hosted Qwen3.5-27B.

Kozuchi Agent construction map: phase-driven agent, reuse orchestration, K candidate trajectories, selector — End-to-end construction map: hidden benchmark tests are used only by the official evaluator, never during selection. — click to open in `figures/`

The same harness transfers from Python to Java unchanged: per-phase scaffold behavior matches the Python run within ±5 percentage points on every phase, and the remaining failures are dominated by semantic correctness, Java-specific harness failures, and selection errors — not proprietary-model advantages.

04 · Results

Results at a glance

browse stats/ ↗

374/500 = 74.80% on SWE-bench Verified (cloud evaluator), Wilson 95% CI [70.82, 78.41]; internal Docker re-grade 376/500 = 75.20%.
Rank 12/135 on the frozen SWE-bench Verified leaderboard — #1 among open-weight agents; McNemar paired tests win significantly against 16/17 open-weight peers (BH-FDR).
41/128 = 32.03% on Multi-SWE-bench Java with the same agent, unchanged — rank 4/42, #1 strict open-weight, 34/41 significant wins.
All 495 persisted trajectories submit a patch; 494/495 patches apply cleanly.

Python vs Java headline rates and per-phase message share — Cross-track transfer: one agent, two languages.

Per-repository resolution heatmap vs peers — Per-repository resolution vs. open-weight peers.

05 · Reproducibility

Reproduce every number

open reproduce.sh ↗

The artifact is self-contained: every numeric value, figure, and table in the paper is regenerable from files inside the repository, and INDEX.md maps each claim to its script, its paper section, and its regenerated output.

$ git clone https://github.com/marscod/kozuchi-agent-artifact
$ cd kozuchi-agent-artifact
$ ./reproduce.sh                # every paper number + figure (stats/, figures/, plots/)
$ ./reproduce.sh --paper        # also rebuild paper/main.pdf
$ ./decompress.sh               # optional: restore 22 GB of raw trajectories from data/archives/

Requirements: Python 3.10+ (scripts/numbers/ needs no third-party packages); matplotlib pandas seaborn numpy openpyxl for figures (auto-handled via uv when available); TeX Live only for --paper.

🗃 Dataset note: the Zenodo record (trajectory datasets for Python and Java) is under review and not yet publicly published. Until the record goes live, use the reviewer preview link and place the downloaded archives under data/archives/.

07 · Citation

Citation

open references.bib ↗

If you build on this artifact, please cite the paper. The DOI is 10.1145/3832783.3834531. A freely available preprint is on arXiv:2608.15579.

ACM Reference Format

Mehdi Bahrami, Kosaku Kimura, Satoshi Munakata, Satoshi Nakashima, Yu Ishikawa, Kosuke Maeda, Nao Soma, Kenichi Kobayashi, Keisuke Miyazaki, Keizo Kato, Shigeki Fukuta, Tatsuo Kumano, Nobutaka Imamura, Kevin Musgrave, Shahbaz Abdul Khader, Kwun Ho Ngan, Joe Townsend, Fayas Asharindavida, Matthieu Parizy, Akira Sakai, Yuma Ichikawa, Yang Zhao, Michiaki Takizawa, Taku Fukui, Hiroki Ohtsuji, Wei-Peng Chen, and Hiromichi Kobashi. 2026. Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair. In Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE '26), October 12–16, 2026, Munich, Germany. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3832783.3834531

BibTeX

@inproceedings{kozuchi2026, author = {Bahrami, Mehdi and Kimura, Kosaku and Munakata, Satoshi and Nakashima, Satoshi and Ishikawa, Yu and Maeda, Kosuke and Soma, Nao and Kobayashi, Kenichi and Miyazaki, Keisuke and Kato, Keizo and Fukuta, Shigeki and Kumano, Tatsuo and Imamura, Nobutaka and Musgrave, Kevin and Abdul Khader, Shahbaz and Ngan, Kwun Ho and Townsend, Joe and Asharindavida, Fayas and Parizy, Matthieu and Sakai, Akira and Ichikawa, Yuma and Zhao, Yang and Takizawa, Michiaki and Fukui, Taku and Ohtsuji, Hiroki and Chen, Wei-Peng and Kobashi, Hiromichi}, title = {Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair}, year = {2026}, isbn = {979-8-4007-2882-2}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3832783.3834531}, doi = {10.1145/3832783.3834531}, booktitle = {Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering}, numpages = {13}, keywords = {AI agent, automated program repair, SWE-bench, experience report}, location = {Munich, Germany}, series = {ASE '26} }

Small model, frontier results

Read the paper

How it works

Results at a glance

Reproduce every number

Artifact map

`INDEX.md`

`notebooks/reproduce.html`

`scripts/`

`data/`

`REFERENCES_INDEX.md`

`paper/`

Citation

ACM Reference Format

BibTeX

The artifact in motion