SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

Ziyi Wang; Nan Jiang; Guang Lin; Qifan Song

title	SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
authors	Ziyi Wang, Nan Jiang, Guang Lin, Qifan Song
date	2026-09-07
venue	arXiv (preprint, not peer reviewed)
id	arXiv:2510.08999
affiliation	Purdue University
upvotes	5 (Hugging Face Daily Papers, 2026-11-01)
links	abs pdf hf code
abstract	Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weight pruning or low-bit quantization individually, often resulting in suboptimal compression rates to preserve acceptable performance drops. We introduce a unified framework for simultaneous pruning and low-bit quantization via Bayesian variational learning (\method), which achieves higher compression rates than prior baselines while maintaining comparable performance. The key idea is to employ a spike-and-slab prior to induce sparsity and model quantized weights using Gaussian Mixture Models (GMMs) to enable low-bit precision. Due to the intractability of the objective involving spike-and-slab priors with GMMs, we derive an efficient approximation that facilitates effective compression with minimal accuracy loss. In theory, we provide a consistent result for our proposed variational approach to a sparse and quantized deep neural network. Extensive experiments on compressing ResNet, BERT-base, Llama3.2, and Qwen2.5 models show that our method achieves higher compression rates than a line of existing methods with comparable performance drops. Project page: https://comeusr.github.io/SQS_Webpage.

cite

@misc{wang2026sqs,
  title={SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions},
  author={Ziyi Wang and Nan Jiang and Guang Lin and Qifan Song},
  year={2026},
  eprint={2510.08999},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2510.08999}
}