benchmarks evals
The 3D Model Had to Name the Distance
Pumpire measures point-to-point scale directly instead of treating depth, camera calibration or point-cloud resemblance as a proxy.
Summary
Pumpire measures point-to-point scale directly instead of treating depth, camera calibration or point-cloud resemblance as a proxy.
Pumpire asks whether a 3D foundation model can estimate the physical distance between two points in reconstructed space. The benchmark contains 100 real scenes, 64 frames per scene and physically measured point pairs, totaling 6,400 frames. Its protocol compares image- and video-level models, with and without depth priors, using metric distance error rather than separate depth or camera-intrinsic scores. The authors evaluate 29 baseline configurations. The benchmark does not establish that any model is metrically reliable in deployment; it creates a direct test for physical scale, a capability that geometric-similarity metrics can obscure.
Why it matters
Pumpire measures point-to-point scale directly instead of treating depth, camera calibration or point-cloud resemblance as a proxy.
Limits and context
- The benchmark does not establish that any model is metrically reliable in deployment; it creates a direct test for physical scale, a capability that geometric-similarity metrics can obscure.
Key claims
Pumpire measures point-to-point scale directly instead of treating depth, camera calibration or point-cloud resemblance as a proxy.
Qualification: The benchmark does not establish that any model is metrically reliable in deployment; it creates a direct test for physical scale, a capability that geometric-similarity metrics can obscure.
Evidence: source-2026-10-11-004
Sources
- arXiv preprint 2610.12423, original v1arXiv / submitting research authors · primary research
Corrections
No corrections have been recorded for this story.