| title | Beneath the Surface: What We Overlook When We Talk About Agent Evolution |
|---|---|
| authors | Xiaobo Wang, Zimu Wang, Siyu Ye, Jiaqi Li, Min Tang, Jun Bai, Yuling Shi, Ruobing Wang, Jingjing Zhang, Chaofan Wang, Kai Wu, Mengnan Qi, Kai Cai, Xiaodong Gu, Qi Liu, Zilong Zheng |
| date | 2026-11-05 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.07624 |
| affiliation | University of Science and Technology of China |
| upvotes | 25 (Hugging Face Daily Papers, 2026-11-11) |
| links | abs pdf hf |
| cite | |
| related | ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling ACC: Compiling Agent Trajectories for Long-Context Training GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis Speculative Sampling in Interactive Video Generation PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop |
| abstract | Agent harness evolution enables recursive self-improvement (RSI) without updating model weights. We examine whether higher benchmark scores yield reliable, efficient, and transferable improvement. We evaluate four methods across harnesses, general and coding tasks. Repeated evaluations, controlled agent expansion, component interventions, and admission replays reveal recurring patterns. All sixteen evolved agents gain previously uncovered tasks while losing previously covered ones. In one general-task case, reliability rises by 5.2 points despite losing ten previously covered tasks. Most reliability gains also increase deployment cost. Specific changes usually outperform generic reminders, but their benefits can change when combined. The highest-scoring agent does not yield the largest mean improvement in any of our four pools, and stricter admission can discard useful alternatives. Transfer selectively preserves gains: coverage usually falls on the general-task target across both harness types and paradigms, but rises on the coding target with the outer-layer-only harness. These findings motivate preservation-aware evaluation, tested decision rules, and explicit applicability conditions. As a constructive follow-up, EvoAudit turns these diagnostics into revisions of select, analyze, mutate, and gate, achieving the best performance. |
bibtex
@misc{wang2026beneath,
title={Beneath the Surface: What We Overlook When We Talk About Agent Evolution},
author={Xiaobo Wang and Zimu Wang and Siyu Ye and Jiaqi Li and Min Tang and Jun Bai and Yuling Shi and Ruobing Wang and Jingjing Zhang and Chaofan Wang and Kai Wu and Mengnan Qi and Kai Cai and Xiaodong Gu and Qi Liu and Zilong Zheng},
year={2026},
eprint={2611.07624},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.07624}
}