chips infrastructure
Nine Times Fewer Visual Tokens Preserve Most of the Score
A 1.6-million-parameter module assigns each image a query-aware token budget without retraining the base multimodal model.
Summary
A 1.6-million-parameter module assigns each image a query-aware token budget without retraining the base multimodal model.
Multi-image questions can drown a multimodal model in visual tokens even when only a few regions matter. The mmVB module allocates a query-conditioned budget across images using cross-image context and does not require access to attention matrices at inference. The authors report ninefold fewer visual input tokens, fivefold faster time to first token and sevenfold fewer prefill operations while retaining 94.4 percent of full-resolution accuracy across eight benchmarks. Those figures are averages over the tested models and tasks, not a universal compression guarantee. The result suggests that multi-image systems should decide how much of each image to carry forward rather than compressing every frame equally.
Why it matters
A 1.6-million-parameter module assigns each image a query-aware token budget without retraining the base multimodal model.
Limits and context
- Multi-image questions can drown a multimodal model in visual tokens even when only a few regions matter.
- The mmVB module allocates a query-conditioned budget across images using cross-image context and does not require access to attention matrices at inference.
- Those figures are averages over the tested models and tasks, not a universal compression guarantee.
Key claims
A 1.6-million-parameter module assigns each image a query-aware token budget without retraining the base multimodal model.
Qualification: Multi-image questions can drown a multimodal model in visual tokens even when only a few regions matter.
Evidence: source-2026-10-14-009
Sources
- arXiv preprint 2610.18858, original v1arXiv / submitting research authors · primary research
Corrections
No corrections have been recorded for this story.