| title | Learning Transferable Dynamics Priors from Action to World Modeling |
|---|---|
| authors | Ze Huang, Jiahui Zhang, Hairuo Liu, Chenxi Zhang, Ran Cheng, Li Zhang |
| date | 2026-06-28 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.29501 |
| upvotes | 3 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| primer | World models |
| abstract | We study action-conditioned world modeling as a scalable way to learn transferable dynamics priors for robot learning. By pretraining a model to predict how actions drive visual scene evolution, the resulting world model captures reusable interaction dynamics beyond appearance-level video generation. Concretely, we pretrain a multi-view interactive base diffusion world model, A2World, on large-scale robot manipulation data with real action annotations. We validate the learned dynamics priors from two complementary perspectives. First, we adapt A2World into a task- or scene-specialized real-world simulator, A2World-sim, whose long-horizon rollouts support simulator-based policy evaluation and scalable what-if analysis by replacing real-robot rollouts with world model rollouts. Second, starting from the same pretrained weights, we adapt A2World into a video-action joint prediction model, A2World-policy, that predicts actions under visual and instruction conditioning. Experiments across simulation benchmarks and real-robot settings demonstrate that action-conditioned world model pretraining yields transferable dynamics priors that benefit both simulator-centric and policy-centric robot learning. |
cite
@misc{huang2026learning,
title={Learning Transferable Dynamics Priors from Action to World Modeling},
author={Ze Huang and Jiahui Zhang and Hairuo Liu and Chenxi Zhang and Ran Cheng and Li Zhang},
year={2026},
eprint={2606.29501},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.29501}
}