safety security
The Old Tool Trace Kept Giving New Orders
Plausible but outdated history flipped nearly a third of decisions a compact model had made correctly on the clean trajectory.
Summary
Plausible but outdated history flipped nearly a third of decisions a compact model had made correctly on the clean trajectory.
A paired benchmark holds current tools, policy, request and correct next action constant while comparing original, polluted and oracle-state histories. On Qwen3-1.7B, misleading history flipped 32.1 percent of decisions that were correct on the original trajectory. A teacher-transfer method raised balanced tool-use accuracy to 87.0 percent and up to 91.9 percent with an 8B teacher, according to the authors; these are benchmark results, not guarantees against stale production context.
Why it matters
Plausible but outdated history flipped nearly a third of decisions a compact model had made correctly on the clean trajectory.
Limits and context
- A teacher-transfer method raised balanced tool-use accuracy to 87.0 percent and up to 91.9 percent with an 8B teacher, according to the authors; these are benchmark results, not guarantees against stale production context.
Key claims
Plausible but outdated history flipped nearly a third of decisions a compact model had made correctly on the clean trajectory.
Qualification: A teacher-transfer method raised balanced tool-use accuracy to 87.0 percent and up to 91.9 percent with an 8B teacher, according to the authors; these are benchmark results, not guarantees against stale production context.
Evidence: source-2026-08-07-005
Sources
- arXiv preprint 2608.06057arXiv · primary research
Corrections
No corrections have been recorded for this story.