| title | iADD: Improving Alignment and Diversity in Diffusion Policy Optimization |
|---|---|
| authors | Ashok Prasad Neupane, Saugat Adhikari, Pramish Paudel, Ajad Chhatkuli, Danda Pani Paudel |
| date | 2026-10-01 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.01789 |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Reinforcement learning based post training of diffusion models, such as Denoising Diffusion Policy Optimization (DDPO), optimizes a reverse diffusion process under a reward function. However, current approaches to reward optimizations do so at the cost of diversity and quality. In this paper, we provide better tradeoffs through careful theoretical considerations and method design. We analyze the theoretical framework and mathematically demonstrate that only-latter timestep updates of diffusion model may be harmful for diversity contrary to the conclusions presented in a previous work. Additionally, we propose an incremental Feynman-Kac training based on strong theoretical foundations in order to achieve the best-yet alignment-diversity tradeoffs. We perform extensive experiments and compare our method against related diffusion policy optimization approaches in three different tasks and also provide strong ablations for each component, thus validating strong performance gains in both alignment and diversity. |
cite
@misc{neupane2026iadd,
title={iADD: Improving Alignment and Diversity in Diffusion Policy Optimization},
author={Ashok Prasad Neupane and Saugat Adhikari and Pramish Paudel and Ajad Chhatkuli and Danda Pani Paudel},
year={2026},
eprint={2610.01789},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.01789}
}