benchmarks evals
A Bumped Camera Cuts Robot Success Nearly in Half
LIBERO-RIG changes hardware and setup while leaving the robot tasks themselves fixed.
Summary
LIBERO-RIG changes hardware and setup while leaving the robot tasks themselves fixed.
Robot policies often meet a different camera mount, table height, controller rate or worn gripper pad at deployment. LIBERO-RIG perturbs eleven such setup axes at five severity levels while holding tasks constant. Across nine vision-language-action models and 500 episodes per condition, pooled success fell from 0.932 in the reference setup to 0.767 at middle severity and 0.509 at the highest. Training on visual perturbations did not solve faults invisible to the camera. These are simulated benchmark conditions, but they expose a deployment gap: robustness to scenes is not the same as robustness to the machine carrying out the policy.
Why it matters
LIBERO-RIG changes hardware and setup while leaving the robot tasks themselves fixed.
Limits and context
- Training on visual perturbations did not solve faults invisible to the camera.
- These are simulated benchmark conditions, but they expose a deployment gap: robustness to scenes is not the same as robustness to the machine carrying out the policy.
Key claims
LIBERO-RIG changes hardware and setup while leaving the robot tasks themselves fixed.
Qualification: Training on visual perturbations did not solve faults invisible to the camera.
Evidence: source-2026-10-14-012
Sources
- arXiv preprint 2610.18843, original v1arXiv / submitting research authors · primary research
Corrections
No corrections have been recorded for this story.