| title | MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations |
|---|---|
| authors | Lorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang, Lukas Cavigelli |
| date | 2026-11-07 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.10649 |
| affiliation | HUAWEI Computing Systems Lab |
| upvotes | 30 (Hugging Face Daily Papers, 2026-11-13) |
| links | abs pdf hf code |
| cite | |
| abstract | Weight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently. MACaQ applies a per-channel scale and a block-Hadamard transform online to each layer's activations, while absorbing the corresponding inverse transforms into its weights. Retaining these transforms online avoids cross-layer dependencies and enables a simple streaming implementation of the quantization process, in which only one layer must reside in memory at a time. To improve over round-to-nearest quantization, we introduce Flip-Refinement, a Hessian-aware coordinate-descent procedure that greedily moves individual FP4 codes to adjacent representable values. Despite executing the activation transforms at inference time, MACaQ adds only 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM. MACaQ reaches lower perplexity than DuQuant++ and MR-GPTQ, and already matches their best perplexity on Llama-3-8B in 11 times less quantization time. We further apply MACaQ without architecture-specific modifications to an image diffusion model, illustrating the generality of its layer-local design and show quantization of a 122B parameter model in 64 minutes, using only 32.5GB GPU RAM. |
bibtex
@misc{muller2026macaq,
title={MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations},
author={Lorenz K. Muller and Philippe Bich and Chiara Boretti and Hyun-Min Chang and Jiawei Zhuang and Lukas Cavigelli},
year={2026},
eprint={2611.10649},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.10649}
}