infrastructure
The Target Model Checks Fewer Bad Branches
ValuSpec filters low-value speculative candidates before expensive verification while preserving the target model's greedy output exactly.
Summary
ValuSpec filters low-value speculative candidates before expensive verification while preserving the target model's greedy output exactly.
Retrieval-based speculative decoding can reuse historical drafts, but larger candidate trees accumulate branches unlikely to survive target-model verification. ValuSpec scores and filters a supplied tree before that expensive step. Its threshold comes from the target model's greedy trajectories on a calibration split, and transfer-based supervision aligns valuation with the target without parameter training. The remaining tree is still verified under an output-identical greedy rule, so accepted tokens match ordinary autoregressive decoding. The authors report up to 2.38-times end-to-end speedup in retrieval-based settings and 2.78-times when attached to EAGLE-3. Those maxima depend on the tested models, tasks, calibration and serving setup; the method does not accelerate every workload by that amount.
Why it matters
ValuSpec filters low-value speculative candidates before expensive verification while preserving the target model's greedy output exactly.
Limits and context
- Those maxima depend on the tested models, tasks, calibration and serving setup; the method does not accelerate every workload by that amount.
Key claims
ValuSpec filters low-value speculative candidates before expensive verification while preserving the target model's greedy output exactly.
Qualification: Those maxima depend on the tested models, tasks, calibration and serving setup; the method does not accelerate every workload by that amount.
Evidence: source-2026-10-25-005
Sources
- arXiv preprint 2610.31338, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.