| title | Long-Horizon RL Needs Muon |
|---|---|
| authors | Kai Ruan, Jinghao Lin, Tao Chen, Ziqi Zhou, Qianshan Wei, Zihe Huang |
| date | 2026-10-14 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.20256 |
| affiliation | Renmin University of China |
| upvotes | 19 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Long-horizon agents must coordinate reasoning, tool use, and environment feedback over extended interactions. Learning from sparse outcome rewards requires weighting trajectories of different lengths and updating the policy while rollouts are still being collected. We examine a single-rollout asynchronous recipe combining Muon, token-averaged loss, truncated importance sampling, and auxiliary length and tool-error penalties. Single-rollout sampling allows asynchronous learning without waiting for multiple trajectories for the same issue. We study optimizer choices and recipe components for repository-level issue resolution across two model families. The evaluated training runs show higher training success with the full recipe than with the baselines. |
cite
@misc{ruan2026long,
title={Long-Horizon RL Needs Muon},
author={Kai Ruan and Jinghao Lin and Tao Chen and Ziqi Zhou and Qianshan Wei and Zihe Huang},
year={2026},
eprint={2610.20256},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.20256}
}