benchmarks evals
The Best Agent Finished the Pieces and Lost the Chain
EarthVerse tests 405 reproducible investigations across 199 documented events and 19 natural-hazard families.
Summary
EarthVerse tests 405 reproducible investigations across 199 documented events and 19 natural-hazard families.
The strongest system reached 84.65 percent mean answer-unit accuracy, but only 34.81 percent Strict@95. The gap shows that agents can complete many local steps without preserving a consistent chain across evidence, units, calculations and physical interpretation.
Why it matters
EarthVerse tests 405 reproducible investigations across 199 documented events and 19 natural-hazard families.
Limits and context
- The strongest system reached 84.65 percent mean answer-unit accuracy, but only 34.81 percent Strict@95.
Key claims
EarthVerse tests 405 reproducible investigations across 199 documented events and 19 natural-hazard families.
Qualification: The strongest system reached 84.65 percent mean answer-unit accuracy, but only 34.81 percent Strict@95.
Evidence: source-2026-08-25-003
Sources
- arXiv preprint 2608.23525arXiv · primary research
Corrections
No corrections have been recorded for this story.