Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Subham Sekhar Sahoo; Lingjie Chen; Khiem Pham; Jonathan Geuter; Chaitanya Dwivedi; Varad Pimpalkhute; Yash Akhauri; Alexander Moreno; Mikhail Yurochkin; Zhenting Wang; Mostafa Elhoushi; Nolan Dey; Shane Bergsma; Joel Hestness; John Thickstun; Eric Xing; Zhengzhong Liu

title

authors

Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham, Jonathan Geuter, Chaitanya Dwivedi, Varad Pimpalkhute, Yash Akhauri, Alexander Moreno, Mikhail Yurochkin, Zhenting Wang, Mostafa Elhoushi, Nolan Dey, Shane Bergsma, Joel Hestness, John Thickstun, Eric Xing, Zhengzhong Liu

date

2026-09-03

venue

arXiv (preprint, not peer reviewed)

arXiv:2609.04010

affiliation

Institute of Foundation Models

upvotes

105 (Hugging Face Daily Papers, 2026-11-01)

links

abs pdf hf code

abstract

Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation. To overcome this bottleneck, we introduce diffusion-augmented LLMs, a new class of models that defines an AR model distribution while using diffusion to draw multiple tokens in parallel from that distribution. We decouple the parameters of these models into two sets: AR weights, trained using the standard NTP objective, and lightweight diffusion weights, trained to generate multiple tokens simultaneously. The diffusion weights are learned through a simple Diffusion Distillation phase that adds negligible overhead to existing LLM training pipelines. We also introduce Ψ-Spec, a family of samplers that enables lossless acceleration and inference-time scaling at a fixed context length. Unlike speculative decoding, our method requires no separate draft model. Unlike diffusion LLMs (d-LLMs), it accelerates generation without sacrificing the quality of the underlying AR model. The resulting models, called Uno, can be trained from scratch or built by augmenting existing open-weight AR LLMs. Uno achieves higher throughput than leading speculative-decoding methods at every evaluated batch size and delivers up to 3times speedups over the base AR model, including at the largest batch size supported by the device. Notably, our 8B Uno model outperforms the leading open d-LLM, the 26B DiffusionGemma, and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning. We release code and checkpoints at: https://s-sahoo.github.io/uno/

@misc{sahoo2026unlocking,
  title={Unlocking Lossless Speedups in LLMs via Discrete Diffusion},
  author={Subham Sekhar Sahoo and Lingjie Chen and Khiem Pham and Jonathan Geuter and Chaitanya Dwivedi and Varad Pimpalkhute and Yash Akhauri and Alexander Moreno and Mikhail Yurochkin and Zhenting Wang and Mostafa Elhoushi and Nolan Dey and Shane Bergsma and Joel Hestness and John Thickstun and Eric Xing and Zhengzhong Liu},
  year={2026},
  eprint={2609.04010},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.04010}
}