infrastructure
The Router Saved the Cache and Spent the Quality
Locality training cut cache misses by as much as 60 percent but failed a preregistered one-percent perplexity limit.
Summary
Locality training cut cache misses by as much as 60 percent but failed a preregistered one-percent perplexity limit.
Measurements of a 235-billion-parameter mixture-of-experts model on an 8 GB GPU exposed a memory-bandwidth wall at 0.44 tokens per second. Smaller preregistered training runs made routing more cacheable, including a 99 percent static-pin hit rate, but every configuration missed the quality gate. A training-free rerouting stack reached about 80 percent fewer misses at up to 3.4 percent perplexity cost, making the tradeoff explicit rather than free.
Why it matters
Locality training cut cache misses by as much as 60 percent but failed a preregistered one-percent perplexity limit.
Limits and context
No additional limitation was separately recorded.
Key claims
Locality training cut cache misses by as much as 60 percent but failed a preregistered one-percent perplexity limit.
Evidence: source-2026-08-20-008
Sources
- arXiv preprint 2608.18261arXiv · primary research
Corrections
No corrections have been recorded for this story.