benchmarks evals
The Object Looks Right Until the Other Views Arrive
3DVerifierBench finds a 21.6-point gap between people and the strongest tested model—and shows why one plausible projection is not enough.
Summary
3DVerifierBench finds a 21.6-point gap between people and the strongest tested model—and shows why one plausible projection is not enough.
A generated 3D object can look convincing from one camera while violating a requirement elsewhere. 3DVerifierBench tests that problem across ten tasks, scoring acceptance decisions, rejection explanations and responses to controlled changes in both specifications and candidate objects. In the authors' human study, the strongest evaluated model, GPT-6 Astra, trailed people by 21.6 percentage points. Errors concentrated around exact values, binding a requirement to the right target and deciding whether an apparent mismatch was merely projection-dependent. The team also built 14,126 training examples with contradiction explanations and linked counterfactuals. In human-labeled pools of sixteen candidates, an oracle choice raised complete factual success from 26.7 to 90 percent. Those are benchmark results, not proof that every 3D generator fails. They do show that visual plausibility and requirement-level correctness are separate jobs—and that reliable verification must see the whole object.
Why it matters
3DVerifierBench finds a 21.6-point gap between people and the strongest tested model—and shows why one plausible projection is not enough.
Limits and context
- Those are benchmark results, not proof that every 3D generator fails.
Key claims
3DVerifierBench finds a 21.6-point gap between people and the strongest tested model—and shows why one plausible projection is not enough.
Qualification: Those are benchmark results, not proof that every 3D generator fails.
Evidence: source-2026-10-17-001
Sources
- arXiv preprint 2610.22360, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.