infrastructure
Six Bits Cut Recurrent-State Memory by More Than Fivefold
STEPQuant assigned precision by error impact and memory lifetime instead of quantizing every state uniformly.
Summary
STEPQuant assigned precision by error impact and memory lifetime instead of quantizing every state uniformly.
STEPQuant targets the fixed-size recurrent states used by Delta-rule linear attention, where rounding errors can persist through later updates. It assigns precision across both time and state structure, fitting key-row and value-column scales to distributions and output impact. On Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct, the authors report near-FP32 accuracy at a nominal six-bit budget and better four-bit results than uniform INT8. Their SGLang implementation compressed recurrent state by more than fivefold and reduced total serving memory by as much as 68.7% on the tested workloads.
Why it matters
STEPQuant assigned precision by error impact and memory lifetime instead of quantizing every state uniformly.
Limits and context
No additional limitation was separately recorded.
Key claims
STEPQuant assigned precision by error impact and memory lifetime instead of quantizing every state uniformly.
Evidence: source-2026-09-30-008
Sources
- arXiv preprint 2609.38169arXiv · primary research
Corrections
No corrections have been recorded for this story.