benchmarks evals
Vision Models Lose the Arrows in Scientific Figures
SciFigBench turns 4,352 method diagrams into 40,670 verified questions about dense content, relations and information flow.
Summary
SciFigBench turns 4,352 method diagrams into 40,670 verified questions about dense content, relations and information flow.
Scientific method figures compress a paper's logic into modules, arrows and labels. SciFigBench evaluates dense perception, visual relations and information-flow tracing using 4,352 figures and 40,670 manually verified question-answer pairs. The authors report that strong vision-language models remain substantially limited on relational understanding. A training-free cueing method that externalizes arrow dependencies improved several tasks. The benchmark probes figure interpretation; it does not assess the truth of the underlying papers.
Why it matters
SciFigBench turns 4,352 method diagrams into 40,670 verified questions about dense content, relations and information flow.
Limits and context
- The authors report that strong vision-language models remain substantially limited on relational understanding.
- The benchmark probes figure interpretation; it does not assess the truth of the underlying papers.
Key claims
SciFigBench turns 4,352 method diagrams into 40,670 verified questions about dense content, relations and information flow.
Qualification: The authors report that strong vision-language models remain substantially limited on relational understanding.
Evidence: source-2026-10-21-010
Sources
- arXiv preprint 2610.27087, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.