benchmarks evals
The Digit Returned as a World the Agent Could Only Glimpse
MNIST-PRO turns digit recognition into sequential exploration with limited lookback to isolate perceptual-state construction.
Summary
MNIST-PRO turns digit recognition into sequential exploration with limited lookback to isolate perceptual-state construction.
Ten multimodal models were asked to inspect digits through partial glimpses while maintaining one of four memory representations: visual history, textual state, a structured grid or a consolidated canvas. Full observability remained easy, but partial views exposed failures to integrate fragments, continued search and revise early beliefs after contradictory evidence. The benchmark deliberately removes much of robotics' control complexity, making it a diagnostic for active sensing and memory rather than a claim about full embodied competence.
Why it matters
MNIST-PRO turns digit recognition into sequential exploration with limited lookback to isolate perceptual-state construction.
Limits and context
No additional limitation was separately recorded.
Key claims
MNIST-PRO turns digit recognition into sequential exploration with limited lookback to isolate perceptual-state construction.
Evidence: source-2026-09-01-007
Sources
- arXiv preprint 2608.31022arXiv · primary research
Corrections
No corrections have been recorded for this story.