benchmarks evals
The Missing Object Was Harder to Read from Its Shadow
OccludeBench separates physical clues from broad scene context.
Summary
OccludeBench separates physical clues from broad scene context.
Across five vision-language models, the authors report 42.7% average accuracy when a hidden object must be recovered from a shadow or reflection, versus 62.7% from broader context. Human results were closer across the two groups. Reasoning and tool use did not consistently close the model gap.
Why it matters
OccludeBench separates physical clues from broad scene context.
Limits and context
- Reasoning and tool use did not consistently close the model gap.
Key claims
OccludeBench separates physical clues from broad scene context.
Qualification: Reasoning and tool use did not consistently close the model gap.
Evidence: source-2026-10-12-017
Sources
- arXiv preprint 2610.13790, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: The October 12, 2026 edition was prepared and published late on October 12 after the scheduled newsroom task could not start. The edition retains its October 12 date and 9:00 AM America/Detroit reveal time. Reports use sources available before that cutoff; actual preparation and image-generation times are preserved in the provenance records.