| title | Unlocking Dense Metric Depth Estimation in VLMs |
|---|---|
| authors | Hanxun Yu, Xuan Qu, Yuxin Wang, Jianke Zhu, Lei ke |
| date | 2026-05-15 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.15876 |
| affiliation | Tencent Hunyuan |
| upvotes | 11 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Vision-Language Models (VLMs) excel at 2D tasks such as grounding and captioning, yet remain limited in 3D understanding. A key limitation is their text-only supervision paradigm, which under-constrains fine-grained visual perception and prevents the recovery of dense geometry. Prior methods either distill geometry from external vision models, introducing error accumulation, or enable direct prediction with inefficient per-pixel query or coarse token-level outputs. In this paper, we propose DepthVLM, a simple yet effective framework that transforms a single VLM into a native dense geometry predictor while preserving its multimodal capability. By attaching a lightweight depth head to the LLM backbone and training under a unified vision-text supervision paradigm with a two-stage schedule, DepthVLM generates full-resolution depth maps alongside language outputs in a single forward pass. We further introduce a unified indoor-outdoor metric depth benchmark in a VLM-compatible format. Experiments show that DepthVLM significantly outperforms existing VLMs with higher inference efficiency, surpasses leading pure vision models, and improves complex 3D spatial reasoning, moving toward a truly unified foundation model. All code and checkpoints will be publicly released. |
cite
@misc{yu2026unlocking,
title={Unlocking Dense Metric Depth Estimation in VLMs},
author={Hanxun Yu and Xuan Qu and Yuxin Wang and Jianke Zhu and Lei ke},
year={2026},
eprint={2605.15876},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.15876}
}