| title | Improved Distributional Diffusion Models |
|---|---|
| authors | Tommaso Martorella, Alexandre Galashov, Felix Krause, Stefan Andreas Baumann, Valentin De Bortoli, Arthur Gretton, Björn Ommer |
| date | 2026-09-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.37147 |
| affiliation | CompVis |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Distributional Diffusion Models (DDMs) replace the standard mean-prediction denoiser with a distributional denoiser trained via a scoring rule objective, learning a stochastic approximation to p(x_1 mid x_t) rather than its conditional mean. However, scaling DDMs to modern image-generation settings faces two obstacles: (i) multi-particle training incurs overhead that scales with the number of particles, (ii) DDMs use globally fixed scoring rule hyperparameters, forcing a single trade-off across sampling budgets. We mitigate these limitations by deferring particle expansion to late transformer layers, and the hyperparameter trade-off by introducing time-dependent scoring rule schedules informed by the dynamical regimes of~Biroli2024. Combined with a DiT-based latent setup, these changes make DDM training practical on class-conditional ImageNet-256^2, achieving 4.48 FID at 4 steps and 2.38 at 50 steps with DiT-XL/2, from a single model trained from scratch in one stage, without a teacher, self-distillation or JVPs. The result is a stochastic few-step generator whose FID does not degrade as the sampling budget grows from 4 to 50 NFE, and the same recipe transfers to text-to-image generation. Code and pre-trained models available at https://github.com/CompVis/iDDM. |
cite
@misc{martorella2026improved,
title={Improved Distributional Diffusion Models},
author={Tommaso Martorella and Alexandre Galashov and Felix Krause and Stefan Andreas Baumann and Valentin De Bortoli and Arthur Gretton and Björn Ommer},
year={2026},
eprint={2609.37147},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.37147}
}