titleAgentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
authorsYu Xia, Zhouhang Xie, Xin Xu, Byungkyu Kang, Prarit Lamba, Xiang Gao, Julian McAuley
date2026-06-02
venuearXiv (preprint, not peer reviewed)
idarXiv:2606.03965
affiliationUniversity of California at San Diego
linksabs pdf hf
abstractLarge language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner's generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS matches full-thinking performance with substantial token savings, and enables controllable accuracy-efficiency trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.

cite

@misc{xia2026agentic,
  title={Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning},
  author={Yu Xia and Zhouhang Xie and Xin Xu and Byungkyu Kang and Prarit Lamba and Xiang Gao and Julian McAuley},
  year={2026},
  eprint={2606.03965},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2606.03965}
}