benchmarks evals
A Plausible Number Pulled Even the Accurate Models Off Course
Fourteen models remained susceptible to anchoring when an initial value arrived through a credible pathway.
Summary
Fourteen models remained susceptible to anchoring when an initial value arrived through a credible pathway.
AnchorBench varies both the route by which an anchor appears and whether the number is relevant. The authors report that plausible anchors generally moved judgments more than irrelevant ones, stronger pathways amplified the effect, and distance from the evidence-supported answer weakened it. Even frontier models above 95 percent accuracy without an anchor were not reliably robust, separating baseline competence from resistance to contextual bias.
Why it matters
Fourteen models remained susceptible to anchoring when an initial value arrived through a credible pathway.
Limits and context
- Even frontier models above 95 percent accuracy without an anchor were not reliably robust, separating baseline competence from resistance to contextual bias.
Key claims
Fourteen models remained susceptible to anchoring when an initial value arrived through a credible pathway.
Qualification: Even frontier models above 95 percent accuracy without an anchor were not reliably robust, separating baseline competence from resistance to contextual bias.
Evidence: source-2026-08-17-011
Sources
- arXiv preprint 2608.14320arXiv · primary research
Corrections
No corrections have been recorded for this story.