chips infrastructure
The Memory Budget Moves with the Crowd
Adaptive DHiSparse changes how much context stays on the GPU as inference load rises and falls.
Summary
Adaptive DHiSparse changes how much context stays on the GPU as inference load rises and falls.
Adaptive DHiSparse combines speculative decoding with a hierarchy of GPU and host memory for long-context serving. It protects a required working set, keeps more history resident when capacity is available and retrieves the rest as needed. The paper reports higher throughput under its tested models, context lengths and latency constraints, including a 100,000-token GLM-5.3 configuration. The engineering point is the changing residency budget: spare memory can reduce fetching at low load, then be released under pressure. Reported gains remain tied to the evaluated serving configurations.
Why it matters
Adaptive DHiSparse changes how much context stays on the GPU as inference load rises and falls.
Limits and context
- Reported gains remain tied to the evaluated serving configurations.
Key claims
Adaptive DHiSparse changes how much context stays on the GPU as inference load rises and falls.
Qualification: Reported gains remain tied to the evaluated serving configurations.
Evidence: source-2026-10-13-008
Sources
- arXiv preprint 2610.13708, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: The October 13, 2026 edition was prepared and published late on October 13 after the scheduled newsroom task could not start. It retains its October 13 edition date and 9:00 AM America/Detroit reveal time. Reports use evidence available before that cutoff; actual preparation and image-generation times are preserved in the provenance records.