TheMachine Press

A daily newspaper for the age of artificial intelligence.

Morning editionPermanent story

infrastructure

The Generator Reuses Its Prefill to Rank Two Hundred Passages

One 9-billion-parameter backbone added about 0.1 GiB per decision skill and outscored a cross-encoder at the reported 43,000-token scale.

Published Updated Story ID: mp-2026-11-01-026
Read the complete editionStory JSON

Summary

One 9-billion-parameter backbone added about 0.1 GiB per decision skill and outscored a cross-encoder at the reported 43,000-token scale.

Retrieval systems often keep separate reranking and answer models resident, duplicating memory and passage processing. Glance adds a small decision adapter and pointer head to a 9-billion-parameter generator, then branches short relevance probes from a shared prefill. After 39 minutes of MS MARCO fine-tuning, it exceeded the reported bge reranker scores on SciFact and NFCorpus. On 300 HotpotQA questions, selected context raised answer F1 by 0.045, and ranking quality continued to improve through 200 candidates—about 43,000 tokens—where it beat the cross-encoder by 0.058 nDCG@10. Evaluation covers one backbone family and three datasets.

Why it matters

One 9-billion-parameter backbone added about 0.1 GiB per decision skill and outscored a cross-encoder at the reported 43,000-token scale.

Limits and context

No additional limitation was separately recorded.

Key claims

  1. One 9-billion-parameter backbone added about 0.1 GiB per decision skill and outscored a cross-encoder at the reported 43,000-token scale.

    Evidence: source-2026-11-01-015

Sources

  1. arXiv preprint 2610.40454, original v1arXiv · primary research

Corrections

No corrections have been recorded for this story.