AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models

Yutong Hu; Jan-Nico Zaech; Nikolay Nikolov; Yuanqi Yao; Sombit Dey; Giuliano Albanese; Renaud Detry; Luc Van Gool; Danda Paudel

title	AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models
authors	Yutong Hu, Jan-Nico Zaech, Nikolay Nikolov, Yuanqi Yao, Sombit Dey, Giuliano Albanese, Renaud Detry, Luc Van Gool, Danda Paudel
date	2026-05-11
venue	arXiv (preprint, not peer reviewed)
id	arXiv:2603.10126
affiliation	Institute for Computer Science, Artificial intelligence and Technology
links	abs pdf hf
primer	Vision-language-action models
abstract	We propose a standalone autoregressive (AR) Action Expert that generates actions as a continuous causal sequence while conditioning on refreshable vision-language prefixes. In contrast to existing Vision-Language-Action (VLA) models and diffusion policies that reset temporal context with each new observation and predict actions reactively, our Action Expert maintains its own history through a long-lived memory and is inherently context-aware. This structure addresses the frequency mismatch between fast control and slow reasoning, enabling efficient independent pretraining of kinematic syntax and modular integration with heavy perception backbones, naturally ensuring spatio-temporally consistent action generation across frames. To synchronize these asynchronous hybrid V-L-A modalities, we utilize a re-anchoring mechanism that mathematically accounts for perception staleness during both training and inference. Experiments on simulated and real-robot manipulation tasks demonstrate that the proposed method can effectively replace traditional chunk-based action heads for both specialist and generalist policies. AR-VLA exhibits superior history awareness and substantially smoother action trajectories while maintaining or exceeding the task success rates of state-of-the-art reactive VLAs. Overall, our work introduces a scalable, context-aware action generation schema that provides a robust structural foundation for training effective robotic policies. Code and Videos available at https://arvla.insait.ai

cite

@misc{hu2026ar,
  title={AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models},
  author={Yutong Hu and Jan-Nico Zaech and Nikolay Nikolov and Yuanqi Yao and Sombit Dey and Giuliano Albanese and Renaud Detry and Luc Van Gool and Danda Paudel},
  year={2026},
  eprint={2603.10126},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2603.10126}
}