TheMachine Press

A daily newspaper for the age of artificial intelligence.

Morning editionPermanent story

open source

One Agent Run Becomes a Corpus Test Suite

A read-only auditing agent found seven Slovak web-corpus issues, then compiled them into native-speaker-checked executable tests.

Published Updated Story ID: mp-2026-10-23-008
Read the complete editionStory JSON

Summary

A read-only auditing agent found seven Slovak web-corpus issues, then compiled them into native-speaker-checked executable tests.

Instead of classifying every document with a language model, researchers used a tool-using agent once as a corpus debugger. Three runs over the 26.5-million-document Slovak slice of FineWeb2 cost $2.21 to $2.84 each and produced seven findings across four phenomena. Native-speaker review supported six. The largest was a machine-generated content farm containing 1,408,437 documents, or 5.32 percent of the corpus. The compiled tests then scanned the full slice in 0.7 seconds without a GPU. A static sampling baseline validated more selectors at lower cost, so the agent's value was in prevalence estimates, explanations and reusable tests rather than unqualified superiority.

Why it matters

A read-only auditing agent found seven Slovak web-corpus issues, then compiled them into native-speaker-checked executable tests.

Limits and context

No additional limitation was separately recorded.

Key claims

  1. A read-only auditing agent found seven Slovak web-corpus issues, then compiled them into native-speaker-checked executable tests.

    Evidence: source-2026-10-23-008

Sources

  1. arXiv preprint 2610.31636, original v1arXiv · primary research

Corrections

No corrections have been recorded for this story.