| title | τ_0-WM: A Unified Video-Action World Model for Robotic Manipulation |
|---|---|
| authors | Pengfei Zhou, Shengcong Chen, Di Chen, Jiaxu Wang, Rongjun Jin, Bingwen Zhu, Yike Pan, Songen Gu, Kuanning Wang, Shufeng Nan, Xingyu Qiu, Chenhao Qiu, Pu Yang, Yunuo Cai, Jianxiong Gao, Yifan Li, Yanwei Fu, Xiangyu Yue, Zhi Chen, Jianlan Luo |
| date | 2026-05-31 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.01027 |
| upvotes | 1 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| primer | World models |
| abstract | Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present τ_0-World Model (τ_0-WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, τ_0-WM provides two complementary interfaces. First, a video action model jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state. Second, an action-conditioned video simulator rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores. The model is trained on approximately 27{,}300 hours of real-robot teleoperation, UMI-style interaction, egocentric human videos, and rollout or failure trajectories using modality-specific supervision masks. At inference time, τ_0-WM uses test-time computation to sample action candidates, rank them with re-denoising consistency, and invoke simulator-based rectification for low-quality candidates. On challenging long-horizon and fine-grained robotic manipulation tasks, τ_0-WM shows superior performance over other relevant baselines. |
cite
@misc{zhou2026wm,
title={τ_0-WM: A Unified Video-Action World Model for Robotic Manipulation},
author={Pengfei Zhou and Shengcong Chen and Di Chen and Jiaxu Wang and Rongjun Jin and Bingwen Zhu and Yike Pan and Songen Gu and Kuanning Wang and Shufeng Nan and Xingyu Qiu and Chenhao Qiu and Pu Yang and Yunuo Cai and Jianxiong Gao and Yifan Li and Yanwei Fu and Xiangyu Yue and Zhi Chen and Jianlan Luo},
year={2026},
eprint={2606.01027},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.01027}
}