benchmarks evals
The Benchmark Changed When Only the Simulated User Changed
UserProxyBench found that agent reward can stay high even when a proxy user breaks its private instructions.
Summary
UserProxyBench found that agent reward can stay high even when a proxy user breaks its private instructions.
UserProxyBench evaluates the language model assigned to play the user in interactive agent tests, scoring adherence to private instructions independently of whether the agent completes the task. With the agent fixed and only the user proxy varied across 375 enterprise tasks, mean reward moved by 15.2 points. The authors found user-specification violations in 24.4% of successful episodes; premature disclosure was the dominant failure, and successful episodes with it used 1.06 fewer tool calls on average. The result shows that task reward can conceal a changed interaction, within the tested tau-bench-derived setting and seven proxy models.
Why it matters
UserProxyBench found that agent reward can stay high even when a proxy user breaks its private instructions.
Limits and context
- With the agent fixed and only the user proxy varied across 375 enterprise tasks, mean reward moved by 15.2 points.
Key claims
UserProxyBench found that agent reward can stay high even when a proxy user breaks its private instructions.
Qualification: With the agent fixed and only the user proxy varied across 375 enterprise tasks, mean reward moved by 15.2 points.
Evidence: source-2026-09-30-002
Sources
- arXiv preprint 2609.38043arXiv · primary research
Corrections
No corrections have been recorded for this story.