titleDreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
authorsDreamX Team, Rui Chen, Xiangxiang Chu, Geng Li, Jifan Li, Qingfeng Shi, Datao Tang, Jing Tang, Jun Wang, Pengfei Zhang
date2026-08-13
venuearXiv (preprint, not peer reviewed)
idarXiv:2608.13489
affiliationalibaba
upvotes54 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
primerWorld models
abstractWe present DreamX-Phi 1.0, an action-conditioned video world model for robotic manipulation that, given an observed frame, a language instruction, and a prescribed action sequence comprising end-effector poses and gripper states, predicts the resulting future observations. Yet realism alone does not guarantee faithfulness: a convincing rollout can still move the wrong arm or lose the manipulated object. To ensure the prediction respects each arm's commanded path, we inject per-arm SE(3) transformations into attention via PRoPE-style geometric encoding, preserving arm identity and rigid-motion structure. Action control alone does not fully constrain scene geometry or the evolution of small manipulated objects. We therefore add a lightweight depth branch for scene-level geometry and use SAM3 masks with a frozen V-JEPA teacher to maintain object consistency throughout grasping. We further distill the multi-step generator into a few-step student via distribution-matching distillation for efficient deployment. At the time of writing, achieves first place on Track~1 and second place on Track~2 of the WorldArena~2.0 Challenge. Our model and code will be publicly available.

cite

@misc{team2026dreamx,
  title={DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation},
  author={DreamX Team and Rui Chen and Xiangxiang Chu and Geng Li and Jifan Li and Qingfeng Shi and Datao Tang and Jing Tang and Jun Wang and Pengfei Zhang},
  year={2026},
  eprint={2608.13489},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2608.13489}
}