robotics
The Robot Knew Which Hand Was Late but Did Not Say It
A vision-language model's internal state encoded the delayed hand nearly perfectly, while its answer depended strongly on how the question was phrased.
Summary
A vision-language model's internal state encoded the delayed hand nearly perfectly, while its answer depended strongly on how the question was phrased.
Bimanual tool use creates failures that single-arm benchmarks miss: one hand can be late, a tool can slip while still held, or two causes can overlap. In simulation, the researchers map which sensing channels can identify each cause and show that some combined failures are not recoverable from passive sensing alone. A repair policy that escalated when the cause was unidentifiable matched a policy given the true cause within two attempts. A tested vision-language model internally represented which hand was late almost perfectly, yet its verbal diagnosis depended on wording. The findings concern the constructed task and traces, not every bimanual system.
Why it matters
A vision-language model's internal state encoded the delayed hand nearly perfectly, while its answer depended strongly on how the question was phrased.
Limits and context
- In simulation, the researchers map which sensing channels can identify each cause and show that some combined failures are not recoverable from passive sensing alone.
- The findings concern the constructed task and traces, not every bimanual system.
Key claims
A vision-language model's internal state encoded the delayed hand nearly perfectly, while its answer depended strongly on how the question was phrased.
Qualification: In simulation, the researchers map which sensing channels can identify each cause and show that some combined failures are not recoverable from passive sensing alone.
Evidence: source-2026-10-29-012
Sources
- arXiv preprint 2610.37533, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.