| title | DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation |
|---|---|
| authors | Jiaxing Li, Kai Zou, Cindy Zhou, Kaichen Huang, Junyao Gao, Zile Wang, Yang Liu, Bin Liu, Bo An, Yangguang Li |
| date | 2026-07-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2607.26811 |
| upvotes | 6 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Existing autoregressive video distillation methods commonly adopt a Distribution Matching Distillation (DMD)-based multi-stage pipeline. However, they typically decouple the initialization and DMD stages -- which then pursue different target distributions -- and judge the intermediate student mainly by visual scores such as VBench. In this paper, we revisit this design from a distributional perspective. Given the mode-seeking nature of the distribution matching loss, a good initialization should match the mode coverage of the target DMD teacher, rather than merely pursuing high quality. To analyze this, we introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. It exposes differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support. Furthermore, even when the target distributions are aligned, DMD's reverse-KL objective can still drive the student toward high-probability teacher regions in late training, reducing coverage and diversity. To address this, we propose joint distillation, which combines DMD's mode-seeking objective with a Consistency Distillation-based mode-covering constraint. Experiments show that our method improves generation quality, coverage, and diversity; notably, even with a Wan-1.3B DMD teacher, it outperforms baselines refined with Wan-14B, underscoring the importance of distributional alignment in autoregressive video distillation. |
cite
@misc{li2026distillalign,
title={DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation},
author={Jiaxing Li and Kai Zou and Cindy Zhou and Kaichen Huang and Junyao Gao and Zile Wang and Yang Liu and Bin Liu and Bo An and Yangguang Li},
year={2026},
eprint={2607.26811},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2607.26811}
}