infrastructure
A Document's Memory Cache Trains Itself Smaller
Latent Compaction freezes model weights and optimizes a document-specific KV cache without seeing the downstream questions it will later answer.
Summary
Latent Compaction freezes model weights and optimizes a document-specific KV cache without seeing the downstream questions it will later answer.
Long prompts enlarge the key-value cache that generation must repeatedly read. Latent Compaction treats that cache as a tiny trainable object: it initializes entries from reconstruction, attention and key diversity, then fits attention outputs and distills behavior from the full cache. Model weights stay frozen, and training uses generic prompts about the document rather than evaluation answers. On one RULER setting, the authors report 96.7 percent accuracy at 32-times compression versus 40.0 percent for KVZip+. Transfer tests span Qwen3 and MiniCPM5. The result is not free compression; it exchanges per-document test-time training for a smaller serving memory.
Why it matters
Latent Compaction freezes model weights and optimizes a document-specific KV cache without seeing the downstream questions it will later answer.
Limits and context
- The result is not free compression; it exchanges per-document test-time training for a smaller serving memory.
Key claims
Latent Compaction freezes model weights and optimizes a document-specific KV cache without seeing the downstream questions it will later answer.
Qualification: The result is not free compression; it exchanges per-document test-time training for a smaller serving memory.
Evidence: source-2026-10-15-011
Sources
- arXiv preprint 2610.20582, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: This October 15 edition was prepared before its scheduled 9:00 AM America/Detroit reveal, but publication was delayed by a production build failure. The original reporting and image-generation timestamps are preserved.