benchmarks evals
Toy Games Predict Real-World Visual Reasoning—Mostly
Game2Real-Bench aligns executable game tasks with real images by reasoning requirement instead of appearance.
Summary
Game2Real-Bench aligns executable game tasks with real images by reasoning requirement instead of appearance.
Game2Real-Bench pairs 1,000 procedurally verified game samples with 400 real-world samples that demand the same kinds of perception, spatial reasoning, constraints and planning. Across 25 vision-language models, game and real-world scores had Pearson and Spearman correlations of 0.935. One-shot game examples improved constraint reasoning for several tested models but hurt fine-grained perception. Correlation does not mean every synthetic skill transfers. The benchmark's useful distinction is that a controllable toy can diagnose a reasoning operation even when its pixels look nothing like the target world.
Why it matters
Game2Real-Bench aligns executable game tasks with real images by reasoning requirement instead of appearance.
Limits and context
- Correlation does not mean every synthetic skill transfers.
Key claims
Game2Real-Bench aligns executable game tasks with real images by reasoning requirement instead of appearance.
Qualification: Correlation does not mean every synthetic skill transfers.
Evidence: source-2026-10-14-014
Sources
- arXiv preprint 2610.18834, original v1arXiv / submitting research authors · primary research
Corrections
No corrections have been recorded for this story.