chips infrastructure
The Recurrent Half Was the Easy Half to Quantize
Minima applies NVFP4 weights and activations to all 496 linear layers of a hybrid 27B model, gates included.
Summary
Minima applies NVFP4 weights and activations to all 496 linear layers of a hybrid 27B model, gates included.
Community recipes had left Gated DeltaNet layers—especially decay and write-strength gates—at higher precision because recurrent error was expected to accumulate. The tested 17.5-GiB recipe instead matched BF16 within seed noise across its suite, improved prefill by 14% to 19% and kept retrieval through 64K. Mechanism studies found block scaling localized outliers, gate nonlinearities compressed error and the delta rule forgot injected disturbances. The checkpoint and claims apply to the evaluated Qwen3.8-27B hybrid.
Why it matters
Minima applies NVFP4 weights and activations to all 496 linear layers of a hybrid 27B model, gates included.
Limits and context
No additional limitation was separately recorded.
Key claims
Minima applies NVFP4 weights and activations to all 496 linear layers of a hybrid 27B model, gates included.
Evidence: source-2026-09-04-015
Sources
- arXiv preprint 2609.04098arXiv · primary research
Corrections
No corrections have been recorded for this story.