| title | VideoMDM: Towards 3D Human Motion Generation From 2D Supervision |
|---|---|
| authors | Amir Mann, Gal Michael Harari, Merav Keidar, Or Litany |
| date | 2026-06-11 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.13364 |
| upvotes | 22 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | We introduce VideoMDM, a diffusion-based framework that trains 3D human motion priors directly from accurate 2D poses extracted from monocular videos, without any 3D ground truth. A pretrained 2D-to-3D lifter provides approximate 3D pose sequences that serve as a noisy teacher: these are diffused, denoised by the model in 3D, and supervised in 2D by reprojecting the prediction and comparing against accurate keypoints. We show that, under mild assumptions, a depth-weighted 2D reprojection loss is equivalent in expectation to direct 3D supervision, and we adapt standard 3D motion regularizers - velocity consistency and over-parameterized representation alignment - to this 2D setting. Unlike methods that lift 2D to 3D only at inference, VideoMDM learns a coherent 3D motion manifold during training. On HumanML3D it nearly closes the gap to fully 3D-supervised MDM (FID 0.88 vs 0.54); On real video datasets Fit3D and NBA the method learns to generate motions consistently preferred by humans, with strong quantitative results. |
cite
@misc{mann2026videomdm,
title={VideoMDM: Towards 3D Human Motion Generation From 2D Supervision},
author={Amir Mann and Gal Michael Harari and Merav Keidar and Or Litany},
year={2026},
eprint={2606.13364},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.13364}
}