| title | SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents |
|---|---|
| authors | Qingyao Li, Wenxiang Jiao, Shuai Shao, Kangning Zhang, Yuan Lu, Yi Guo, Weiwen Liu, Weinan Zhang, Yong Yu |
| date | 2026-08-19 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.18852 |
| affiliation | Shanghai Jiao Tong University |
| upvotes | 9 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills. |
cite
@misc{li2026skillgate,
title={SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents},
author={Qingyao Li and Wenxiang Jiao and Shuai Shao and Kangning Zhang and Yuan Lu and Yi Guo and Weiwen Liu and Weinan Zhang and Yong Yu},
year={2026},
eprint={2608.18852},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.18852}
}