benchmarks evals
The Model Followed Recency More Than Reliability
A controlled benchmark made textual summaries, numerical series and external forecasts disagree on purpose.
Summary
A controlled benchmark made textual summaries, numerical series and external forecasts disagree on purpose.
Because the synthetic risk trajectories identify which evidence source matches the ground truth, the study can vary modality, recency, stated reliability and provenance independently. Open-weight instruction models showed systematic text-versus-number preferences and followed recent evidence more consistently than reliability labels, sometimes over-weighting an external forecast against direct context. The finding isolates a heuristic failure mode for tool-augmented decisions rather than measuring a deployed domain.
Why it matters
A controlled benchmark made textual summaries, numerical series and external forecasts disagree on purpose.
Limits and context
No additional limitation was separately recorded.
Key claims
A controlled benchmark made textual summaries, numerical series and external forecasts disagree on purpose.
Evidence: source-2026-08-23-016
Sources
- arXiv preprint 2608.20116arXiv · primary research
Corrections
No corrections have been recorded for this story.