infrastructure
The State-Space Server Remembered Inputs Instead of Rewriting State
ReplaySSM separates output computation from periodic checkpoint updates.
Summary
ReplaySSM separates output computation from periodic checkpoint updates.
State-space models keep compact recurrent state, but moving that state on every decoding step can become expensive. ReplaySSM represents it with a checkpoint plus recent inputs, avoiding writes on most steps and making speculative rollback a buffer-pointer operation. Implemented in vLLM for Mamba-2 and Gated DeltaNet, it accelerated standard decoding by 1.20–1.47 times at batch size 256 across four hybrid models. On two models in the 120-billion-parameter class, speculative throughput was about 1.8 times the vLLM baseline. Separate checkpoint precisions also reduced the measured effects of quantization error.
Why it matters
ReplaySSM separates output computation from periodic checkpoint updates.
Limits and context
No additional limitation was separately recorded.
Key claims
ReplaySSM separates output computation from periodic checkpoint updates.
Evidence: source-2026-10-12-012
Sources
- arXiv preprint 2610.13778, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: The October 12, 2026 edition was prepared and published late on October 12 after the scheduled newsroom task could not start. The edition retains its October 12 date and 9:00 AM America/Detroit reveal time. Reports use sources available before that cutoff; actual preparation and image-generation times are preserved in the provenance records.