benchmarks evals
A Blind Control Beat the Adaptive Layer Program
Multiple-choice headroom survived even when perturbations ignored the input, exposing option order as a confound.
Summary
Multiple-choice headroom survived even when perturbations ignored the input, exposing option order as a confound.
Across two models and 4,413 items, input-blind controls produced more oracle headroom than the real layer programs in the primary shared-order comparisons. Rotating answer options sharply reduced both. The authors argue that selection gain alone does not identify a computation-specific benefit.
Why it matters
Multiple-choice headroom survived even when perturbations ignored the input, exposing option order as a confound.
Limits and context
- The authors argue that selection gain alone does not identify a computation-specific benefit.
Key claims
Multiple-choice headroom survived even when perturbations ignored the input, exposing option order as a confound.
Qualification: The authors argue that selection gain alone does not identify a computation-specific benefit.
Evidence: source-2026-10-08-020
Sources
- arXiv preprint 2610.10368arXiv · primary research
Corrections
No corrections have been recorded for this story.