open source
Reproduction Improves When Execution Knowledge Survives
Refined ML artifacts lift four coding agents above 90 percent mean success while narrowing the gap between models.
Summary
Refined ML artifacts lift four coding agents above 90 percent mean success while narrowing the gap between models.
ReproBridge packages a refined codebase, environment setup, experiment launchers and a reproduction guide after an agent reconstructs a machine-learning experiment. Its benchmark contains 1,029 tasks from 387 experiments across 36 papers, each anchored by five human-verified reference scores. In fresh sessions, refined artifacts raised average reproduction success across four models from 76.4 to 92.6 percent, cut downstream API spending 19.7 percent and narrowed the cross-model success spread from 21.6 to 3.9 points. The study argues for preserving recovered procedure, not repeatedly paying agents to infer it.
Why it matters
Refined ML artifacts lift four coding agents above 90 percent mean success while narrowing the gap between models.
Limits and context
- The study argues for preserving recovered procedure, not repeatedly paying agents to infer it.
Key claims
Refined ML artifacts lift four coding agents above 90 percent mean success while narrowing the gap between models.
Qualification: The study argues for preserving recovered procedure, not repeatedly paying agents to infer it.
Evidence: source-2026-11-08-009
Sources
- arXiv preprint 2611.08452, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.