| title | Decoding Without a Decoder: Direct Pixel Prediction from Diffusion Transformers |
|---|---|
| authors | Hansheng Chen, Shengqu Cai, Jan Ackermann, Rayhan Zirvi, Gordon Wetzstein, Xinchen Yan, Leonidas Guibas |
| date | 2026-11-17 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.22287 |
| affiliation | Stanford University |
| upvotes | 11 (Hugging Face Daily Papers, 2026-11-21) |
| links | abs pdf hf |
| cite | |
| related | Thinking Inertia: LLMs Keep Thinking When Told Not To Hybrid Reasoning Agents AISPA: User-Centric System Prompt Auditing for Large Language Model Applications The Last Human-Written Paper: Agent-Native Research Artifacts Stateless Language Agents: Scaling Long-Horizon Automated Research |
| abstract | Latent diffusion models rely on a decoder to map denoised latents to pixels, adding complexity and inference cost. We introduce Decoding Without a Decoder (DWD), which lets a finetuned diffusion transformer produce final pixels directly through a linear head. Unlike previous methods, where the denoising output leads to final pixels, DWD separates the two roles: the original denoising head continues to drive sampling, then the new DWD head predicts the pixels at the final step. We extend pretrained latent models with DWD efficiently by finetuning only the last few transformer blocks, training the DWD head with pixel quality losses while anchoring the denoising head to the pretrained model to preserve the sample trajectory. Across latent image and video generators, DWD removes the decoder and reduces inference cost while maintaining or even improving pixel quality. On Wan video generation, it visibly reduces detail distortion and flickering, lowering warping error by 23%. Remarkably, the same separation also improves pixel diffusion models that do not even use a decoder: a final-step DWD head produces substantially better texture detail in text-to-image generation, and improves ImageNet FID from 1.55 to 1.43, a new state of the art among pixel flow models. |
bibtex
@misc{chen2026decoding,
title={Decoding Without a Decoder: Direct Pixel Prediction from Diffusion Transformers},
author={Hansheng Chen and Shengqu Cai and Jan Ackermann and Rayhan Zirvi and Gordon Wetzstein and Xinchen Yan and Leonidas Guibas},
year={2026},
eprint={2611.22287},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.22287}
}