titleRoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
authorsTianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, Songling Liu, Haowen Yan, Honghao Su, Zhiyang Dou, Kaixuan Wang, Dandan Zhang, Yunze Liu, Yan Qin, Qiwei Liang, Qiwei Wu, Zijian Lin, Wenwei Lin, Yuran Wang, Minghua He, Tianshu Wu, Ruihai Wu, Jingquan Zhou, Kai-Chong Lei, Haibao Yu, Yuanfeng Ji, Weiyang Jin, Guanyu Lin, Xiaofan Li, Qi Xiong, Renjing Xu, Zhongyu Li, Wenhao Chai, Enze Xie, Ziwei Wang, Yao Mu, Hao Dong, Wojciech Matusik, Mingyu Ding, Wenbo Ding, Ping Luo, Masayoshi Tomizuka
date2026-07-07
venuearXiv (preprint, not peer reviewed)
idarXiv:2607.04434
affiliationRoboDojo-Benchmark
upvotes16 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractGeneralist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-horizon, or skill-narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulation enables scalable feedback but misses physical deployment challenges, while real-world evaluation is costly, time-consuming, and difficult to reproduce. We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities. The simulation benchmark evaluates five dimensions: generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following, while the real-world benchmark exposes policies to challenging physical-world deployment conditions. RoboDojo supports scalable evaluation through heterogeneous parallel simulation in Isaac Sim and provides RoboDojo-RealEval, a reproducible real-world evaluation system with remote cloud access, standardized hardware, scene reset, evaluation protocol, and deployment interface. Together with XPolicyLab, policies can be integrated once and evaluated across simulation and real-world settings with minimal adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard and systematic analysis of current policy performance. The website is available at http://robodojo-benchmark.com/.

cite

@misc{chen2026robodojo,
  title={RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies},
  author={Tianxing Chen and Yue Chen and Zixuan Li and Junyuan Tang and Kailun Su and Haoran Lu and Weijie Wan and Baijun Chen and Songling Liu and Haowen Yan and Honghao Su and Zhiyang Dou and Kaixuan Wang and Dandan Zhang and Yunze Liu and Yan Qin and Qiwei Liang and Qiwei Wu and Zijian Lin and Wenwei Lin and Yuran Wang and Minghua He and Tianshu Wu and Ruihai Wu and Jingquan Zhou and Kai-Chong Lei and Haibao Yu and Yuanfeng Ji and Weiyang Jin and Guanyu Lin and Xiaofan Li and Qi Xiong and Renjing Xu and Zhongyu Li and Wenhao Chai and Enze Xie and Ziwei Wang and Yao Mu and Hao Dong and Wojciech Matusik and Mingyu Ding and Wenbo Ding and Ping Luo and Masayoshi Tomizuka},
  year={2026},
  eprint={2607.04434},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2607.04434}
}