infrastructure
Model Weights and Attention States Share the Same Pages
Rimuru dynamically reallocated memory across tensor types, reporting up to 19.03 times baseline throughput and 98.1 percent lower average time to first token.
Summary
Rimuru dynamically reallocated memory across tensor types, reporting up to 19.03 times baseline throughput and 98.1 percent lower average time to first token.
The vLLM-based system combines tensor-specific memory views, a split-and-merge allocator and fused mixture-of-experts access so heterogeneous tensors can reuse one elastic pool.
Why it matters
Rimuru dynamically reallocated memory across tensor types, reporting up to 19.03 times baseline throughput and 98.1 percent lower average time to first token.
Limits and context
No additional limitation was separately recorded.
Key claims
Rimuru dynamically reallocated memory across tensor types, reporting up to 19.03 times baseline throughput and 98.1 percent lower average time to first token.
Evidence: source-2026-11-01-020
Sources
- arXiv preprint 2610.40305, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.