benchmarks evals
The Agent Match Stopped When the Evidence Was Ready
Variance reduction plus continuously monitored confidence sequences cut the hands needed for a precise comparison.
Summary
Variance reduction plus continuously monitored confidence sequences cut the hands needed for a precise comparison.
Across 15 poker-agent configurations and 71,439 paired hands, AIVAT reduced outcome variance by a median factor of 54; paired with an asymptotic confidence sequence, raw outcomes needed a median 74 times as many hands to reach the same one-big-blind precision. Exact finite-sample stopping requires a justified payoff bound, which the authors establish structurally for Leduc hold'em.
Why it matters
Variance reduction plus continuously monitored confidence sequences cut the hands needed for a precise comparison.
Limits and context
No additional limitation was separately recorded.
Key claims
Variance reduction plus continuously monitored confidence sequences cut the hands needed for a precise comparison.
Evidence: source-2026-08-08-017
Sources
- arXiv preprint 2608.06362arXiv · primary research
Corrections
No corrections have been recorded for this story.