| title | Progressive Agent Skill Generation via Reinforcement Learning |
|---|---|
| authors | Junhao Shen, Zhanqiu Zhang, Yiwen Guo, Hong Cheng |
| date | 2026-08-03 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.01678 |
| affiliation | The Chinese University of Hong Kong - Database Group |
| upvotes | 52 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Existing skill generation methods largely rely on heuristics or pipeline-style consolidation, which must be specially designed for different evidence sources. In contrast, learning-based approaches offer a more unified way to model skill generation across heterogeneous sources. However, learning-based skill generation remains challenging because skills lack a natural supervision signal based on relevance or correctness; their value can largely be determined only by whether they improve the behavior of the agent on downstream tasks. To address this challenge, we propose Skill-α, a reinforcement learning method for progressively generating high-quality agent skills. Specifically, we formulate skill generation as a sequential editing process that decomposes skill construction into individually evaluable edits, and introduce a novel rollback reward that evaluates each edit by comparing downstream execution under the original and edited skills on an anchored query. Extensive experiments show that Skill-α generates more effective skills than methods based on heuristics or pipelines in both document-to-skill and experience-to-skill settings. Under the main GPT-4o worker, Skill-α improves average downstream success rates over the strongest skill-generation baseline by 3.3 points on CL-Bench and 6.7 points on tau2-bench. Further ablations validate the importance of rollback reward and progressive generation. |
cite
@misc{shen2026progressive,
title={Progressive Agent Skill Generation via Reinforcement Learning},
author={Junhao Shen and Zhanqiu Zhang and Yiwen Guo and Hong Cheng},
year={2026},
eprint={2608.01678},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.01678}
}