| title | Breaking Diffusion: Parallel Reasoning with Draft Refinement |
|---|---|
| authors | Jaesang Yu, Seoyeon Byun, Yun Wang, Yan Lu, Juho Kim |
| date | 2026-10-19 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.26542 |
| affiliation | Microsoft |
| upvotes | 22 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Drafting an answer allows a reasoning model to reconsider each decision in the context of the others. Yet repeated diffusion denoising does not ensure that individually plausible token updates form a better answer together. We introduce Draft Refinement Language Models (DRLMs), which use a single denoiser to propose parallel revisions and evaluate a candidate draft. We train the denoiser to reconstruct every token from uniformly corrupted drafts, teaching both preservation and repair. Its predictions specify revisions directly, without a diffusion reverse transition. Evaluation requires a different use of these predictions: confidence in an existing token reflects both contextual support and evidence from observing the token itself. Using the known corruption process, we correct for this self-token evidence to estimate how well each token is supported by the prompt and the remaining draft. Aggregating these estimates yields a cavity energy computed in one forward pass, without an additional scoring model. Each round proposes a revision, reads the completed candidate in its revised context, and retains it only if its energy is lower. Across Countdown, Sudoku-Extreme, and Maze-Hard, our method outperforms autoregressive and discrete diffusion baselines. With 6M parameters, DRLM reaches 44.4% on Countdown-5 and 93.9% on Sudoku-Extreme. The results establish complete-draft parallel refinement as an effective alternative to reverse diffusion for complex reasoning. |
cite
@misc{yu2026breaking,
title={Breaking Diffusion: Parallel Reasoning with Draft Refinement},
author={Jaesang Yu and Seoyeon Byun and Yun Wang and Yan Lu and Juho Kim},
year={2026},
eprint={2610.26542},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.26542}
}