benchmarks evals
The Best Data Agent Excelled on Only a Third of Tasks
Argo-Bench hides ground truth behind 235 tables and grades decisions by simulated consequences.
Summary
Argo-Bench hides ground truth behind 235 tables and grades decisions by simulated consequences.
Argo-Bench simulates a 2024 New York food-delivery platform at scale, exports it as a 235-table, 7.5-billion-row ERP warehouse and asks agents to reconstruct facts before actions such as fraud bans, incentives or back pay. Each of its 210 tasks has an executable reference solution. The strongest of 14 tested frontier and open-weight models scored at least 95 on only 34.8 percent of tasks and averaged 59.5 points. The warehouse is synthetic but designed from public data, literature and filings.
Why it matters
Argo-Bench hides ground truth behind 235 tables and grades decisions by simulated consequences.
Limits and context
- The strongest of 14 tested frontier and open-weight models scored at least 95 on only 34.8 percent of tasks and averaged 59.5 points.
Key claims
Argo-Bench hides ground truth behind 235 tables and grades decisions by simulated consequences.
Qualification: The strongest of 14 tested frontier and open-weight models scored at least 95 on only 34.8 percent of tasks and averaged 59.5 points.
Evidence: source-2026-10-02-009
Sources
- arXiv preprint 2610.02122arXiv · primary research
Corrections
No corrections have been recorded for this story.