frontier models
Production Traces Became an Agent’s Skill Library
Two enterprise benchmarks showed that the best way to mine procedural knowledge depends on the work domain.
Summary
Two enterprise benchmarks showed that the best way to mine procedural knowledge depends on the work domain.
Skill-mining systems increasingly turn execution traces into reusable instructions, yet production logs may not reliably say which runs succeeded. The study held its mining pipeline fixed while varying three evidence regimes—successes only, labeled successes and failures, or the same mixed traces without labels—and two skill forms, ordered workflows or declarative ontologies. On ThinkingBox-Bench, workflows held a small advantage and hiding outcomes hurt; APEX-Agents instead showed a moderate preference for ontologies and no clear evidence-regime winner. The authors' conclusion resists a universal recipe: task structure determines whether an agent benefits more from procedure, vocabulary or outcome labels.
Why it matters
Two enterprise benchmarks showed that the best way to mine procedural knowledge depends on the work domain.
Limits and context
- Skill-mining systems increasingly turn execution traces into reusable instructions, yet production logs may not reliably say which runs succeeded.
- The study held its mining pipeline fixed while varying three evidence regimes—successes only, labeled successes and failures, or the same mixed traces without labels—and two skill forms, ordered workflows or declarative ontologies.
Key claims
Two enterprise benchmarks showed that the best way to mine procedural knowledge depends on the work domain.
Qualification: Skill-mining systems increasingly turn execution traces into reusable instructions, yet production logs may not reliably say which runs succeeded.
Evidence: source-2026-10-06-010
Sources
- arXiv preprint 2610.05777arXiv · primary research
Corrections
No corrections have been recorded for this story.