chips infrastructure
Four-Bit Matrix Math Was Waiting on the Softmax
A hardware-aware FlashAttention path maps scores directly to FP4 probabilities and carries forward quantization into the backward pass.
Summary
A hardware-aware FlashAttention path maps scores directly to FP4 probabilities and carries forward quantization into the backward pass.
Shrinking matrix products on Blackwell tensor cores exposes conversion and on-chip dependency costs elsewhere in attention. Direct-P reached up to 2.13 times BF16 forward throughput for noncausal inference on an NVIDIA GB200. A causal training route reused quantized queries and keys with FP8 gradient operands, accelerating a complete single-GPU 8B update by up to 1.14 times. In matched distributed training, every tested MXFP4 probability-and-value trajectory diverged, sharply limiting the claim.
Why it matters
A hardware-aware FlashAttention path maps scores directly to FP4 probabilities and carries forward quantization into the backward pass.
Limits and context
No additional limitation was separately recorded.
Key claims
A hardware-aware FlashAttention path maps scores directly to FP4 probabilities and carries forward quantization into the backward pass.
Evidence: source-2026-09-04-014
Sources
- arXiv preprint 2609.04105arXiv · primary research
Corrections
No corrections have been recorded for this story.