A Thousand Papers Become Places to Experiment

Scientific papers are readable records, but they rarely arrive as standardized environments in which a research agent can change a method and measure the result. Scientist World uses an autonomous pipeline to extract an open research question from each paper and reproduce the paper’s method as a baseline. The authors release the first 1,000 verified environments plus 6,400 hours of recorded research traces. On their evaluation, frontier research agents surpassed the reproduced human baseline in 74 percent of environments, while training on the traces raised a smaller model’s ideation score by 158 percent to match the frontier systems tested. Those percentages depend on the benchmark’s reproduction and scoring choices; they do not mean agents surpassed scientists generally. The release matters because it turns a claim about automated research into a large, inspectable experimental substrate.