titleπ^4: A Foundation Model for 4D Scene Geometry and Motion
authorsYifan Wang, Weiguang Zhao, Ni Yang, Haoyu Guo, Junting Dong, Weinan Zhang, Chunhua Shen
date2026-11-17
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.23144
affiliationShanghai AI Laboratory
upvotes13 (Hugging Face Daily Papers, 2026-11-19)
linksabs pdf hf code
cite
relatedNCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction
Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning
Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flattening
Sage-Taiyi: A Metacognitive Architecture for Scientific Discovery
Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation
abstractπ^4 is a unified foundation model for 4D scene perception. From diverse, uncalibrated RGB observations in the wild---a single image, an unordered collection, or a video sequence---it recovers dense geometry, camera poses, and metric scale, together with 2D/3D point tracks and moving-object masks for multi-image inputs. It couples a geometry branch with a geometry-conditioned motion branch, which predicts image-space correspondences that are lifted into 3D using reconstructed target-view geometry. Across diverse benchmarks, π^4 achieves the best overall average rank in geometry evaluation, alongside competitive performance in single- and multiview 2D/3D point tracking and moving-object segmentation. With a shared backbone, the full geometry-and-motion system contains approximately 0.5B parameters---less than half the size of most baselines. It produces joint reconstructions and tracks in one forward pass, including large-scale scenes with thousands of input images despite training with contexts of at most 64 views. We further present the data and training recipe underlying these capabilities, together with empirical studies of key design choices. Code and models are available at https://github.com/yyfz/Pi4.

bibtex

@misc{wang2026foundation,
  title={π^4: A Foundation Model for 4D Scene Geometry and Motion},
  author={Yifan Wang and Weiguang Zhao and Ni Yang and Haoyu Guo and Junting Dong and Weinan Zhang and Chunhua Shen},
  year={2026},
  eprint={2611.23144},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.23144}
}