robotics
Preferences Pulled a Robot Beyond Its Original Behavior
PrefPI iterated on relative judgments of self-generated trajectories instead of requiring demonstrations of the target behavior.
Summary
PrefPI iterated on relative judgments of self-generated trajectories instead of requiring demonstrations of the target behavior.
PrefPI treats preferred trajectories as a conditional generative distribution and amplifies their density ratio against the broader behavior prior through classifier-free guidance. Repeating that step lets a pretrained generative policy move toward behaviors that were rare or absent initially. On real hardware, the authors report increasing object transport height from 10.7 to 19.8 centimeters using 150 preference-labeled trajectories. That measured shift demonstrates the reported task, not unconstrained out-of-distribution control.
Why it matters
PrefPI iterated on relative judgments of self-generated trajectories instead of requiring demonstrations of the target behavior.
Limits and context
- That measured shift demonstrates the reported task, not unconstrained out-of-distribution control.
Key claims
PrefPI iterated on relative judgments of self-generated trajectories instead of requiring demonstrations of the target behavior.
Qualification: That measured shift demonstrates the reported task, not unconstrained out-of-distribution control.
Evidence: source-2026-10-01-008
Sources
- arXiv preprint 2609.40165arXiv · primary research
Corrections
No corrections have been recorded for this story.