A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM

Xiaoang Xu; Siyuan Liu; Shuo Wang; Junlan Feng; Fanyu Meng; Zhu Zhang; Jixun Wang; Xiaorong Wang; Zihan Zhou; Xin Li; Chaojun Xiao; Yiming Zhang; Huijia Wu; Liuyu Xiang; Peipei Li; Zhaofeng He

title

authors

Xiaoang Xu, Siyuan Liu, Shuo Wang, Junlan Feng, Fanyu Meng, Zhu Zhang, Jixun Wang, Xiaorong Wang, Zihan Zhou, Xin Li, Chaojun Xiao, Yiming Zhang, Huijia Wu, Liuyu Xiang, Peipei Li, Zhaofeng He

date

2026-09-07

venue

arXiv (preprint, not peer reviewed)

arXiv:2609.07821

upvotes

9 (Hugging Face Daily Papers, 2026-11-01)

links

abs pdf hf

abstract

Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs. Existing methods either lose intermediate information through hard pruning or lack a principled criterion for continuous compression. We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture. After projecting question, step, and solution representations into a 3D PCA space, it measures alignment between each local transition and global question-to-solution direction. Aligned steps remain explicit text, whereas deviating steps are compressed into continuous latent tokens. Directional angles capture both local semantics and reasoning dynamics: small angles indicate direct execution and answer formation, while large angles more frequently involve checking, correction, and branch exploration; their temporal variation reveals exploration, convergence, and refinement stages. To train this architecture, we introduce stepwise embedding forcing, which pools each redundant step into a single latent embedding, and label forcing, which supervises that latent token with a soft multi-modal vocabulary distribution instead of a hard one-hot label. Experiments on Qwen3.5-9B and Qwen3.6-27B across six in-domain and out-of-domain benchmarks show that A*-Thought-V2 improves average accuracy by up to 2.6% while reducing response length by up to half, increasing Accuracy per Computation Unit by 2.29times, and reducing preprocessing and training time by 94.6% and up to 80.3%, respectively. Representation analyses suggest that latent states form a compact region distinct from textual states, while higher entropy at latent-token positions reflects broader soft targets that encourage richer step-level feature learning.

@misc{xu2026thought,
  title={A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM},
  author={Xiaoang Xu and Siyuan Liu and Shuo Wang and Junlan Feng and Fanyu Meng and Zhu Zhang and Jixun Wang and Xiaorong Wang and Zihan Zhou and Xin Li and Chaojun Xiao and Yiming Zhang and Huijia Wu and Liuyu Xiang and Peipei Li and Zhaofeng He},
  year={2026},
  eprint={2609.07821},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.07821}
}