| title | SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation |
|---|---|
| authors | Nadun Ranawaka, Josiah Wong, Wei-Lin Pai, Wei-Teng Chu, Tianyuan Dai, Masoud Moghani, Hang Yin, Yunfan Jiang, Wesley Durbano, Brandon Huynh, Yu Fang, Linxi Fan, Danfei Xu, Ruohan Zhang, Li Fei-Fei, Bowen Wen, Ajay Mandlekar, Yuke Zhu |
| date | 2026-06-26 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.28276 |
| affiliation | NVIDIA |
| upvotes | 16 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Training and evaluating robot policies in the real world is costly and difficult to scale. We introduce SimFoundry, a modular and automated system for zero-shot real-to-sim scene construction from a video. SimFoundry generates sim-ready digital twins and supports object, scene, and task editing, enabling the automated generation of diverse digital cousins: affordance-preserving variations of reconstructed real-world scenes. Policies trained on SimFoundry data transfer zero-shot to challenging real tasks involving multi-step manipulation, articulated object interaction, and bimanual interaction, and its digital cousins (variations of the original scene, objects, and tasks) facilitate generalization to new real-world conditions. Across 7 manipulation tasks and 5 policy architectures, SimFoundry simulation evaluations strongly predict real-world performance, with mean Pearson correlation 0.911 and mean maximum ranking violation 0.018. When evaluating sim-trained policies zero-shot in the real world, policies trained with object, scene, and task cousins in simulation show average task success rate improvements of 17%, 21%, and 40%, respectively. Additional details at https://research.nvidia.com/labs/gear/simfoundry/ . |
cite
@misc{ranawaka2026simfoundry,
title={SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation},
author={Nadun Ranawaka and Josiah Wong and Wei-Lin Pai and Wei-Teng Chu and Tianyuan Dai and Masoud Moghani and Hang Yin and Yunfan Jiang and Wesley Durbano and Brandon Huynh and Yu Fang and Linxi Fan and Danfei Xu and Ruohan Zhang and Li Fei-Fei and Bowen Wen and Ajay Mandlekar and Yuke Zhu},
year={2026},
eprint={2606.28276},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.28276}
}