benchmarks evals
The Attribution Method Mostly Found Large Gradients
BehaviorTrace planted a known learned behavior and found that simple confounds often matched targeted rollout-attribution scores.
Summary
BehaviorTrace planted a known learned behavior and found that simple confounds often matched targeted rollout-attribution scores.
BehaviorTrace asks which online reinforcement-learning rollout taught a model a known planted behavior. Across three Qwen2.5-1.5B seeds, a control that ranked training steps only by gradient magnitude reached 4.2 to 4.5 times chance and matched or beat the best targeted estimator on two seeds. At saturated checkpoints, fluency predicted the behavior label at least as well as every compared gradient method. After controlling for fluency, per-rollout findings changed across seeds and generation draws; only trigger-token gradient alignment held consistently. The study proposes no replacement estimator. Its contribution is a stricter evaluation harness and a warning that plausible attribution maps can be driven by update size or fluency rather than the behavior being traced.
Why it matters
BehaviorTrace planted a known learned behavior and found that simple confounds often matched targeted rollout-attribution scores.
Limits and context
- Across three Qwen2.5-1.5B seeds, a control that ranked training steps only by gradient magnitude reached 4.2 to 4.5 times chance and matched or beat the best targeted estimator on two seeds.
- After controlling for fluency, per-rollout findings changed across seeds and generation draws; only trigger-token gradient alignment held consistently.
Key claims
BehaviorTrace planted a known learned behavior and found that simple confounds often matched targeted rollout-attribution scores.
Qualification: Across three Qwen2.5-1.5B seeds, a control that ranked training steps only by gradient magnitude reached 4.2 to 4.5 times chance and matched or beat the best targeted estimator on two seeds.
Evidence: source-2026-10-08-016
Sources
- arXiv preprint 2610.10422arXiv · primary research
Corrections
No corrections have been recorded for this story.