infrastructure
A Looped Model Reuses the Final Cache Earlier
BeNice trains recurrent passes to share one key-value cache, cutting simulated long-context memory pressure without losing reported validation quality.
Summary
BeNice trains recurrent passes to share one key-value cache, cutting simulated long-context memory pressure without losing reported validation quality.
Stochastic-depth pretraining first pushes recurrent representations toward a common limit; post-training then broadcasts the final recurrence’s cache into the first. In simulated low-memory serving, sharing improved cache hit rate, throughput and tail time to first token. The figures depend on the tested architectures and simulation.
Why it matters
BeNice trains recurrent passes to share one key-value cache, cutting simulated long-context memory pressure without losing reported validation quality.
Limits and context
No additional limitation was separately recorded.
Key claims
BeNice trains recurrent passes to share one key-value cache, cutting simulated long-context memory pressure without losing reported validation quality.
Evidence: source-2026-10-24-020
Sources
- arXiv preprint 2610.31633, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.