safety
The Agent Computed the Right Number and Cited the Wrong Evidence
Retrieving a simulator report from files changed which value some models trusted even when the underlying computation stayed fixed.
Summary
Retrieving a simulator report from files changed which value some models trusted even when the underlying computation stayed fixed.
Using deterministic PHREEQC geochemistry reports and 158 execution-verified questions, researchers tested the common practice of storing long tool output outside the context window. Under indexed retrieval, two GPT models lost 11.3 and 7.9 accuracy points while two Claude models showed no detectable cost. Replaying 11,007 simulator calls showed that the GPT models often computed the keyed value but answered from higher-precision files and misrounded. Naming the authoritative output recovered 6.3 to 7.9 points for three GPT models. The finding is harness-specific, but it shows why a correct tool call does not guarantee an evidence-grounded answer.
Why it matters
Retrieving a simulator report from files changed which value some models trusted even when the underlying computation stayed fixed.
Limits and context
- The finding is harness-specific, but it shows why a correct tool call does not guarantee an evidence-grounded answer.
Key claims
Retrieving a simulator report from files changed which value some models trusted even when the underlying computation stayed fixed.
Qualification: The finding is harness-specific, but it shows why a correct tool call does not guarantee an evidence-grounded answer.
Evidence: source-2026-10-20-015
Sources
- arXiv preprint 2610.26836, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.