| title | Agent-G^2: Gaussian Guidance for Agentic Reinforcement Learning |
|---|---|
| authors | Zixuan Wang, Yanrui Miao, Zhengxi Lu, Teng Pan, Yiwen Qiu, Hongxing Li, Peng Qiu, Ruiqing Zhang, Yongliang Shen |
| date | 2026-08-24 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.23318 |
| affiliation | Zhejiang University |
| upvotes | 23 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Hint-based reinforcement learning addresses reward sparsity in long-horizon agentic tasks by retaining a prefix of an expert trajectory before each rollout, letting the policy explore from a state closer to success. Its effectiveness hinges on the guidance depth: how much of the trajectory to keep. Existing methods treat this depth as a deterministic scalar. Scheduled approaches share one value across samples and ignore per-task heterogeneity; per-sample probing estimates it separately at the cost of extra rollouts. We find that useful guidance occupies a band of depths whose informativeness profile is approximately Gaussian around the band center, rather than concentrating at a single optimal point. We propose Agent-G^2, a Gaussian guidance framework that draws the depth per task from a Gaussian whose center and spread are estimated online from rollouts already collected for policy optimization, requiring no probe rollouts or learned depth predictor. The center combines a global baseline with per-cluster difficulty, and the spread tracks within-cluster variance. We evaluate Agent-G^2 on ALFWorld and WebShop on Qwen2.5-1.5B / 7B-Instruct. Agent-G^2 outperforms the strongest hint-based, hint-free, and Aux-RL baselines on ALFWorld by 2.3 / 3.9 / 7.4 points at under one-third the rollout cost of per-sample probing. |
cite
@misc{wang2026agent,
title={Agent-G^2: Gaussian Guidance for Agentic Reinforcement Learning},
author={Zixuan Wang and Yanrui Miao and Zhengxi Lu and Teng Pan and Yiwen Qiu and Hongxing Li and Peng Qiu and Ruiqing Zhang and Yongliang Shen},
year={2026},
eprint={2608.23318},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.23318}
}