| title | EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video |
|---|---|
| authors | Hyunjin Kim, Ri-Zhao Qiu, Guangqi Jiang, Xiaolong Wang |
| date | 2026-06-15 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.16202 |
| affiliation | University of California at San Diego |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Humans naturally understand object physics through everyday interactions, but faithfully predicting complex deformable dynamics, such as elastic materials and fabrics, remains a major challenge for computer vision and robotics. We present EgoPhys, a framework that constructs deformable physical digital twins from egocentric RGB-only video using generalizable priors. EgoPhys overcomes the limitations of existing methods to enable controllable deformable digital twin generation from egocentric videos by distilling per-object inverse-physics solutions into a compact codebook, enabling prediction of dense spring stiffness fields for unseen objects without per-spring test-time optimization. Trained with generalizable priors from diverse egocentric interactions, EgoPhys outperforms baselines in reconstruction, future prediction, and zero-shot generalization. To support training and evaluation, we curate an egocentric interaction dataset covering diverse deformable objects, scenes, and manipulation styles. We deploy EgoPhys on a real xArm6 robot, demonstrating that a digital twin initialized from a single egocentric human play video can serve as an internal world representation to aid in deformable-object planning, highlighting egocentric RGB observations as a scalable path toward real-to-sim pipelines. |
cite
@misc{kim2026egophys,
title={EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video},
author={Hyunjin Kim and Ri-Zhao Qiu and Guangqi Jiang and Xiaolong Wang},
year={2026},
eprint={2606.16202},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.16202}
}