titleMACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
authorsLorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang, Lukas Cavigelli
date2026-11-07
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.10649
affiliationHUAWEI Computing Systems Lab
upvotes30 (Hugging Face Daily Papers, 2026-11-13)
linksabs pdf hf code
cite
abstractWeight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently. MACaQ applies a per-channel scale and a block-Hadamard transform online to each layer's activations, while absorbing the corresponding inverse transforms into its weights. Retaining these transforms online avoids cross-layer dependencies and enables a simple streaming implementation of the quantization process, in which only one layer must reside in memory at a time. To improve over round-to-nearest quantization, we introduce Flip-Refinement, a Hessian-aware coordinate-descent procedure that greedily moves individual FP4 codes to adjacent representable values. Despite executing the activation transforms at inference time, MACaQ adds only 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM. MACaQ reaches lower perplexity than DuQuant++ and MR-GPTQ, and already matches their best perplexity on Llama-3-8B in 11 times less quantization time. We further apply MACaQ without architecture-specific modifications to an image diffusion model, illustrating the generality of its layer-local design and show quantization of a 122B parameter model in 64 minutes, using only 32.5GB GPU RAM.

bibtex

@misc{muller2026macaq,
  title={MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations},
  author={Lorenz K. Muller and Philippe Bich and Chiara Boretti and Hyun-Min Chang and Jiawei Zhuang and Lukas Cavigelli},
  year={2026},
  eprint={2611.10649},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.10649}
}