titleHarness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement
authorsHaoyang Yan, Min-le Su, Hangfan Zhang, Zhanhao Li, Chen Zhang, Shao Zhang, Yang Chen, Lei Bai, Shuyue Hu
date2026-09-01
venuearXiv (preprint, not peer reviewed)
idarXiv:2609.01481
upvotes20 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractThis paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and usable software systems without human intervention. We introduce Harness-of-Harness (HoH), a framework that enables coding agents to continually improve software during autonomous development. HoH operates on existing coding-agent harnesses, and organizes their executions into iterative planning-coding-testing loops. To sustain improvement across loops, HoH balances repair with capability growth, scopes development into small and verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing agent workflows. It progressively exposes deliverables, role-specific tools, and skills, encourages reuse rather than recreation, and maintains versioned project histories. On GameCraft-Bench, FrontierSWE, and ProgramBench, three harness-model pairs (Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3), HoH consistently outperforms the corresponding standalone harnesses, achieving an average relative gain of 52.25 percent and a maximum gain of 82.86 percent after three iterations. In a multi-day deployment with more than 70 iterations, HoH autonomously develops a first-person-shooter game, featuring a coherent storyline, fully implemented core mechanics, human-playable experience, polished visuals and integrated audio. Github: https://github.com/Flesymeb/HarnessOfHarness Project Page: https://flesymeb.github.io/HarnessOfHarness/

cite

@misc{yan2026harness,
  title={Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement},
  author={Haoyang Yan and Min-le Su and Hangfan Zhang and Zhanhao Li and Chen Zhang and Shao Zhang and Yang Chen and Lei Bai and Shuyue Hu},
  year={2026},
  eprint={2609.01481},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.01481}
}