| title | Video Generative Models as Geometry Learner |
|---|---|
| authors | Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, Jiankang Deng |
| date | 2026-08-28 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.28549 |
| upvotes | 14 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation. Leveraging off-the-shelf image diffusion models, they either (i) train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity of exploring the intrinsic correlation of these geometric targets, or (ii) jointly fine-tune modified image diffusion backbones (e.g., altered self-attention), which typically demands substantial labeled data. To overcome these limitations in a principled fashion, we repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated innovatively as a next-frames prediction task. Our method, GeoNeXt, inherits naturally structured knowledge and richer priors from the video model, while further adapting them for joint modeling of images and geometry targets (image <-> geometry), enabling more data efficient and effective learning of geometry. Extensive experiments validate our method for zero-shot monocular depth and surface normal estimation across diverse datasets, outperforming both previous task-specific and unified generative competitors while using substantially less training data. Notably, our method rivals discriminative state-of-the-art approaches trained on over 100x more data and even standouts on several benchmarks. |
cite
@misc{yang2026video,
title={Video Generative Models as Geometry Learner},
author={Haosen Yang and Jifei Song and Zhensong Zhang and Xiatian Zhu and Jiankang Deng},
year={2026},
eprint={2608.28549},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.28549}
}