infrastructure
Two-Bit Weights Met the Cost of Their Decoder
A full multi-shell Leech-lattice decoder exposes the gap between compact on-disk weights and the bytes a serving kernel actually reads.
Summary
A full multi-shell Leech-lattice decoder exposes the gap between compact on-disk weights and the bytes a serving kernel actually reads.
The implementation expands a 301-class codebook into GPU layouts and fuses dequantization with matrix-vector multiplication for batch-one decoding. Binary bit planes beat one-hot masks, but the served layout still read 2.40× more bytes and ran 2.27× slower than the compared trellis kernel. With a shared output head, end-to-end gains rose from 1.11× at 4B to 1.41× at 14B, while the 4B quality cost remained substantial, underscoring that nominal bit rate is not the whole serving story.
Why it matters
A full multi-shell Leech-lattice decoder exposes the gap between compact on-disk weights and the bytes a serving kernel actually reads.
Limits and context
- With a shared output head, end-to-end gains rose from 1.11× at 4B to 1.41× at 14B, while the 4B quality cost remained substantial, underscoring that nominal bit rate is not the whole serving story.
Key claims
A full multi-shell Leech-lattice decoder exposes the gap between compact on-disk weights and the bytes a serving kernel actually reads.
Qualification: With a shared output head, end-to-end gains rose from 1.11× at 4B to 1.41× at 14B, while the 4B quality cost remained substantial, underscoring that nominal bit rate is not the whole serving story.
Evidence: source-2026-09-03-008
Sources
- arXiv preprint 2609.02652arXiv · primary research
Corrections
No corrections have been recorded for this story.