research
Only Eighty-Five Studies Made It Through the Artifact Door
A six-stage audit fully or partially reproduced 6.52 percent of 1,304 eligible neuro-symbolic AI studies from their published artifacts.
Summary
A six-stage audit fully or partially reproduced 6.52 percent of 1,304 eligible neuro-symbolic AI studies from their published artifacts.
The authors began with 5,497 records, deduplicated and screened them, then sought verifiable public code for 1,304 eligible studies. They report that 849 had no qualifying code artifact; 455 entered the artifact inventory and bounded rerun, where 85 studies were fully or partially reproduced. Missing non-code artifacts blocked 321 attempts, while missing or unusable repositories blocked 42. The audit measures one subfield through its stated protocol, not reproducibility across all AI research, but it shows why a code-available label alone does not complete an experimental record.
Why it matters
A six-stage audit fully or partially reproduced 6.52 percent of 1,304 eligible neuro-symbolic AI studies from their published artifacts.
Limits and context
- The audit measures one subfield through its stated protocol, not reproducibility across all AI research, but it shows why a code-available label alone does not complete an experimental record.
Key claims
A six-stage audit fully or partially reproduced 6.52 percent of 1,304 eligible neuro-symbolic AI studies from their published artifacts.
Qualification: The audit measures one subfield through its stated protocol, not reproducibility across all AI research, but it shows why a code-available label alone does not complete an experimental record.
Evidence: source-2026-08-30-001
Sources
- arXiv preprint 2608.26236arXiv · primary research
Corrections
No corrections have been recorded for this story.