benchmarks evals
One Prompt Turned 18.6 Into 5.4
OV3D-Bench separated localization from semantics and found that small prompt changes could collapse open-vocabulary 3D detection.
Summary
OV3D-Bench separated localization from semantics and found that small prompt changes could collapse open-vocabulary 3D detection.
Across seven indoor and outdoor datasets, the benchmark removed per-image category oracles and scored localization, semantic robustness and transfer separately. Detectors often placed a box correctly but chose an adjacent label; one model’s AP fell from 18.6 to 5.4 when its prompt expanded from “car” to a detailed photographic phrase. The study also found a target-aware protocol inflated another model’s ScanNet result by 1.9 times, suggesting geometry is ahead of open-vocabulary semantics.
Why it matters
OV3D-Bench separated localization from semantics and found that small prompt changes could collapse open-vocabulary 3D detection.
Limits and context
No additional limitation was separately recorded.
Key claims
OV3D-Bench separated localization from semantics and found that small prompt changes could collapse open-vocabulary 3D detection.
Evidence: source-2026-08-19-012
Sources
- arXiv preprint 2608.17110arXiv · primary research
Corrections
No corrections have been recorded for this story.