chips infrastructure
Four-Bit Inference Lost Its Calibration Dataset
DualQuant derives activation scaling from the weight-reconstruction problem.
Summary
DualQuant derives activation scaling from the weight-reconstruction problem.
Activation quantization usually needs calibration examples even when compressing the weights does not. DualQuant uses scales obtained while minimizing weight-reconstruction error to rescale activations as well. It supports several low-bit formats, including MX and NVFP4, and the authors report matching or exceeding SmoothQuant accuracy across most tested models and formats. A packed four-bit weight-and-activation path for Llama-3.1-8B on an RTX 4090 achieved up to 2.61 times lower memory use and 2.46 times higher throughput than BF16. Those maxima belong to that evaluated implementation, rather than every model or accelerator.
Why it matters
DualQuant derives activation scaling from the weight-reconstruction problem.
Limits and context
- Activation quantization usually needs calibration examples even when compressing the weights does not.
Key claims
DualQuant derives activation scaling from the weight-reconstruction problem.
Qualification: Activation quantization usually needs calibration examples even when compressing the weights does not.
Evidence: source-2026-10-12-008
Sources
- arXiv preprint 2610.13767, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: The October 12, 2026 edition was prepared and published late on October 12 after the scheduled newsroom task could not start. The edition retains its October 12 date and 9:00 AM America/Detroit reveal time. Reports use sources available before that cutoff; actual preparation and image-generation times are preserved in the provenance records.