| title | Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent |
|---|---|
| authors | Run Luo, Kuan Zhang, Qi Deng, Yukun Chen, Zhihao Yang, Yufei Zhang, Renrong Hu, Yi Zhang, Guiyu Ma, Tianyu Fan, Longze Chen, Hao Sun, Jiaming Li, Jiaming Yang, Xiaohao Liu, Tao Tan, Minzheng Wang, Hangtian Jia, Shuai Yuan, Lvdi Wang, Xiao Liu, Chao Ma, Yuting Zou, Yinhe Zheng, Xiaobo Xia, Dawei Jiang, Shaoqing Ren |
| date | 2026-11-13 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.18397 |
| upvotes | 128 (Hugging Face Daily Papers, 2026-11-18) |
| links | abs pdf hf code |
| cite | |
| primer | Vision-language-action models |
| abstract | We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds. |
bibtex
@misc{luo2026gamer,
title={Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent},
author={Run Luo and Kuan Zhang and Qi Deng and Yukun Chen and Zhihao Yang and Yufei Zhang and Renrong Hu and Yi Zhang and Guiyu Ma and Tianyu Fan and Longze Chen and Hao Sun and Jiaming Li and Jiaming Yang and Xiaohao Liu and Tao Tan and Minzheng Wang and Hangtian Jia and Shuai Yuan and Lvdi Wang and Xiao Liu and Chao Ma and Yuting Zou and Yinhe Zheng and Xiaobo Xia and Dawei Jiang and Shaoqing Ren},
year={2026},
eprint={2611.18397},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.18397}
}