| title | MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation |
|---|---|
| authors | Ishaan Preetam Chandratreya, David Charatan, Basile Van Hoorick, Sergey Zakharov, Vitor Guizilini, Phillip Isola, Vincent Sitzmann |
| date | 2026-06-08 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.09056 |
| upvotes | 8 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Video generative models have become increasingly powerful, but long-range consistency remains challenging to achieve because even a few dozen frames require impractically long transformer sequence lengths. We show that this issue can be mitigated by generating video using coarse-to-fine rollout within a multi-scale token space. Our approach is simple: first, we pre-train an autoencoder that compresses each frame into a hierarchy of tokens, with levels ranging from the typical latent resolution to only a handful of tokens per frame. The coarsest levels capture the most consequential information, such as scene layout and semantics, while finer levels add high-frequency appearance and texture. Then, we train a video diffusion model to generate these tokens using coarse-to-fine rollout. By carefully controlling the level of detail at which frames are generated and used as context during each rollout step, we are able to preserve long-range consistency in geometry and object permanence while spending less compute on the long-range consistency of less perceptually relevant details. We validate this approach using a custom dataset of long Minecraft videos, where it produces substantially more consistent rollouts compared to existing baselines. |
cite
@misc{chandratreya2026millivid,
title={MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation},
author={Ishaan Preetam Chandratreya and David Charatan and Basile Van Hoorick and Sergey Zakharov and Vitor Guizilini and Phillip Isola and Vincent Sitzmann},
year={2026},
eprint={2606.09056},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.09056}
}