frontier models
Geometry Moves Into the Attention Coordinates
DK-VLM keeps semantic attention intact while a second kernel tracks 3D points, camera pose and reference-frame changes.
Summary
DK-VLM keeps semantic attention intact while a second kernel tracks 3D points, camera pose and reference-frame changes.
Spatial reasoning across camera views requires more than attaching reconstructed features to visual tokens. DK-VLM adds a geometric attention kernel that uses estimated 3D points and relative camera transformations, then learns a gate that combines it with the pretrained model’s semantic kernel. Reference-Frame Composition supervision teaches successive heading and turn updates. The 8-billion-parameter model posted the highest reported average across six compared spatial benchmarks, including 74.33 on VSI-Bench and 58.28 on ReVSI-32, while retaining general video understanding. The scores are author-reported and do not show that reconstructed geometry is always accurate.
Why it matters
DK-VLM keeps semantic attention intact while a second kernel tracks 3D points, camera pose and reference-frame changes.
Limits and context
- The scores are author-reported and do not show that reconstructed geometry is always accurate.
Key claims
DK-VLM keeps semantic attention intact while a second kernel tracks 3D points, camera pose and reference-frame changes.
Qualification: The scores are author-reported and do not show that reconstructed geometry is always accurate.
Evidence: source-2026-10-18-004
Sources
- arXiv preprint 2610.22356, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.