| title | RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion Models |
|---|---|
| authors | Xing Cong, Hanlin Tang, Kan Liu, Lan Tao, Lin Qu, Chenhao Xie |
| date | 2026-05-26 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.26632 |
| affiliation | RTP-LLM |
| upvotes | 7 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs. While prior work has reduced this cost via quantization and distillation, semi-structured sparsity, which can nearly halve FLOPs, remains underexplored. A key reason is that most existing approaches focus on weight sparsification, and pruning 50% of the weights can remove critical model capacity and degrade generation quality. Our study, however, shows that DiT activations are intrinsically sparse and significantly more robust to N:M semi-structured sparsification than weights. Motivated by this observation, we advocate a paradigm shift from weight sparsification to activation sparsification. We propose RT-Lynx, which applies N:M sparsification to activations and incorporates error-compensation techniques to mitigate accuracy loss. We further implement highly optimized CUDA kernels tailored to this setting, achieving up to a 1.55x speedup on average in linear layers. Extensive experiments across multiple diffusion models demonstrate that our method preserves the generation quality of the original models while substantially accelerating inference. |
cite
@misc{cong2026rt,
title={RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion Models},
author={Xing Cong and Hanlin Tang and Kan Liu and Lan Tao and Lin Qu and Chenhao Xie},
year={2026},
eprint={2605.26632},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.26632}
}