| title | Experience-Handoff Reinforcement Learning for Recursive Self-Improvement |
|---|---|
| authors | Doyoung Kim, Byeongguk Jeon, Kailin Zhang, Sherry Yang |
| date | 2026-11-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.16409 |
| affiliation | KAIST AI |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-15) |
| links | abs pdf hf code |
| cite | |
| related | OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs Language Models Can Control Their Own Attention OmniRetrieval: Unified Retrieval across Heterogeneous Knowledge Sources GRACE: Generation-aware latent compression for efficient video generation |
| abstract | It is common to train Large Language Models (LLMs) for agentic tasks with Reinforcement Learning (RL) from scalar rewards. However, this throws away the fine-grained information about the environment that the agent experienced during the trial (e.g., error logs). To utilize this rich information, recent works wrap the agent in a harness that refines its own inputs through reflection or compaction and hands past experience to future rollouts, but the construction of the handed-off artifacts (e.g., a playbook) is itself not learned. We want the best of both worlds: gradient updates from a scalar reward, while also utilizing environment information to hand off experience. As a concrete testbed, we fix the Agentic Context Engineering () harness and train one model to act, reflect, and curate task-local playbooks. Our method, , evaluates candidate experience-handoff artifacts by their downstream effect on future actor rollouts, and co-optimizes the role-specific behaviors (action generator, insight reflector, and playbook curator) under a fixed scaffold. Across agentic benchmarks, achieves higher held-out performance than baseline methods. The results suggest that agent training should optimize not only the actions in the current rollout, but also the model-written experience that future rollouts inherit. |
bibtex
@misc{kim2026experience,
title={Experience-Handoff Reinforcement Learning for Recursive Self-Improvement},
author={Doyoung Kim and Byeongguk Jeon and Kailin Zhang and Sherry Yang},
year={2026},
eprint={2611.16409},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.16409}
}