benchmarks evals
Vision Models Lost to Rules in Social Navigation Scenes
SocialNav-SUB tested spatial, temporal and social reasoning against human and rule-based baselines.
Summary
SocialNav-SUB tested spatial, temporal and social reasoning against human and rule-based baselines.
The SocialNav-SUB benchmark asks vision-language models questions about real social-navigation scenes, including agent relationships and likely human intentions. Across the tested systems, the best model's agreement with human answers remained below both human consensus and a simpler rule-based approach. The result identifies a gap between general visual-language competence and the scene understanding needed for socially compliant robot motion. It evaluates perception and reasoning, not complete autonomous navigation safety.
Why it matters
SocialNav-SUB tested spatial, temporal and social reasoning against human and rule-based baselines.
Limits and context
- It evaluates perception and reasoning, not complete autonomous navigation safety.
Key claims
SocialNav-SUB tested spatial, temporal and social reasoning against human and rule-based baselines.
Qualification: It evaluates perception and reasoning, not complete autonomous navigation safety.
Evidence: source-2026-10-01-006
Sources
- arXiv preprint 2609.40245arXiv · primary research
Corrections
No corrections have been recorded for this story.