| title | VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation |
|---|---|
| authors | Yihong Luo, Tianyu Pang, Tianyang Hu |
| date | 2026-11-04 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.06796 |
| upvotes | 12 (Hugging Face Daily Papers, 2026-11-06) |
| links | abs pdf hf code |
| cite | |
| abstract | Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student's changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student's sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference. |
bibtex
@misc{luo2026va,
title={VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation},
author={Yihong Luo and Tianyu Pang and Tianyang Hu},
year={2026},
eprint={2611.06796},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.06796}
}