infrastructure
Sparse Attention Stops Materializing the Score Table
HyScale finds exact top-k indices on the fly and replays selection, reporting a 2.07-times training speedup for million-token GLM-5.1 runs.
Summary
HyScale finds exact top-k indices on the fly and replays selection, reporting a 2.07-times training speedup for million-token GLM-5.1 runs.
Learned sparse attention can still score every preceding token before reading only a small selected set, making selection itself the bottleneck. HyScale treats the result as indices rather than an activation tensor. HyTopK finds each query's exact top-k without materializing the full score table, and selection replay reuses the forward-pass decision during recomputation. The authors report a 2.07-times GLM-5.1 training speedup at one million tokens on 512 H20 GPUs, plus layer-level gains on two other systems. Those numbers depend on the named hardware, models and baselines; the contribution is a system design for exact selection reuse, not a universal doubling of long-context training speed.
Why it matters
HyScale finds exact top-k indices on the fly and replays selection, reporting a 2.07-times training speedup for million-token GLM-5.1 runs.
Limits and context
- Learned sparse attention can still score every preceding token before reading only a small selected set, making selection itself the bottleneck.
- Those numbers depend on the named hardware, models and baselines; the contribution is a system design for exact selection reuse, not a universal doubling of long-context training speed.
Key claims
HyScale finds exact top-k indices on the fly and replays selection, reporting a 2.07-times training speedup for million-token GLM-5.1 runs.
Qualification: Learned sparse attention can still score every preceding token before reading only a small selected set, making selection itself the bottleneck.
Evidence: source-2026-10-26-009
Sources
- arXiv preprint 2610.32057, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.