| title | AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video |
|---|---|
| authors | Jiaming Tan, Mingliang Zhai, Zhen Li, Yuwei Wu, Chuanhao Li, Kaipeng Zhang |
| date | 2026-09-13 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.14462 |
| affiliation | Alaya Lab |
| upvotes | 27 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| primer | World models |
| abstract | Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a representation trade-off: perspective models operate on local views and must preserve off-screen content over long rollouts, whereas broader spatial coverage is typically obtained by synthesizing full-sphere videos or constructing explicit 3D representations. Motivated by the complementary roles of global context and selective local acuity in visual perception, we present AlayaVista, a camera-controllable streaming video world model that decouples panoramic world evolution from perspective observation synthesis. Given a single perspective image, AlayaVista constructs a 360-degree scene prior using a pretrained panorama expansion model and then evolves the scene as a camera-conditioned panoramic latent state. A latent viewport renderer maps this state to the requested perspective video latents, while a perspective refiner restores details, suppresses artifacts, and performs super-resolution. To support efficient streaming, we adapt the panoramic generator to chunk-autoregressive generation and distill both panoramic generation and perspective refinement into few-step processes. To provide the supervision required by this design, we construct MUGEN, a large-scale real-world panoramic video dataset containing 1,318 hours of videos at resolutions of at least 4K, together with rich semantic and geometric annotations. |
cite
@misc{tan2026alayavista,
title={AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video},
author={Jiaming Tan and Mingliang Zhai and Zhen Li and Yuwei Wu and Chuanhao Li and Kaipeng Zhang},
year={2026},
eprint={2609.14462},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.14462}
}