titleTuringSpace: Grounding Spatial Intelligence in Action with Interactive CAPTCHAs
authorsXuwei Ding, Skylar Zhai, Linxin Song, Jiarui Feng, Dongqi Fu, Huajun Xi, Junnan Li, Jingcheng Liang, Jieyu Zhao, Yinglong Xia, Hong Li, Jian Kang
date2026-11-19
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.25387
affiliationMeta
upvotes22 (Hugging Face Daily Papers, 2026-11-21)
linksabs pdf hf code
cite
relatedSharpening Tax in Post-Training
World Action Modeling with Progressive Visual Planning
Native Action-Prior Learning from Videos for World Action Models
InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants
Context Language Models
abstractHumans act in the visual world by inferring the 3D layout behind a 2D view, predicting how an action changes it, and correcting themselves after seeing the result. Can multimodal large language models (MLLMs) do the same? CAPTCHAs offer a natural test, since humans solve them through this very loop, clicking and dragging on the screen and adjusting after each result. Yet existing CAPTCHA benchmarks are largely solved and rarely spatial, while spatial benchmarks rarely require action. We therefore present TuringSpace, a synthetic web environment of 125 interactive CAPTCHAs that span five spatial abilities and must be solved through clicks and drags on the screen. Every task is a program that generates new instances with a checker and an oracle, so the tasks also form a scalable training gym. Humans solve 94.1% of the tasks, whereas the best model, GPT-6 Astra, still fails 25.9% of them, and the best open-weight model, Kimi K3, fails 68.0%. Our further analysis shows that most failures come from misreading the screenshot, and that model reasoning rarely corrects a wrong belief but hallucinates evidence to defend it. To test whether such spatial abilities can be learned, we fine-tune Qwen3.5-9B on this gym. Its success rate rises from 1.6% to 33.1%, above Kimi K3, and it improves on all seven out-of-distribution benchmarks, including a 9.9-point gain on the real-world indoor videos of VSI-Bench. These results show that grounding spatial intelligence in screen actions exposes what MLLMs lack and offers a scalable way to teach it, with gains that transfer to real-world scenes. We will release TuringSpace to support further study.

bibtex

@misc{ding2026turingspace,
  title={TuringSpace: Grounding Spatial Intelligence in Action with Interactive CAPTCHAs},
  author={Xuwei Ding and Skylar Zhai and Linxin Song and Jiarui Feng and Dongqi Fu and Huajun Xi and Junnan Li and Jingcheng Liang and Jieyu Zhao and Yinglong Xia and Hong Li and Jian Kang},
  year={2026},
  eprint={2611.25387},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.25387}
}