benchmarks evals
Theoretical Computer Science Gets an Executable Exam
TCSExam mixes theorem proving, solver engineering, constructions and open-conjecture work behind kernel or task-checker verification.
Summary
TCSExam mixes theorem proving, solver engineering, constructions and open-conjecture work behind kernel or task-checker verification.
The full benchmark contains 100 expert-reviewed tasks across four tracks; a 20-task Lite version cuts attempts by 80 percent while preserving a reported rank correlation of 0.93 across shared agents. GPT-6 Astra led the full index at 71.9 and Opus 5.5 led Lite at 72.3. The evaluation produced checked solver outputs, constructions and special-case bounds, not resolutions of theoretical computer science as a field. Its contribution is an extensible, executable test bed with sources and Docker recipes.
Why it matters
TCSExam mixes theorem proving, solver engineering, constructions and open-conjecture work behind kernel or task-checker verification.
Limits and context
- The evaluation produced checked solver outputs, constructions and special-case bounds, not resolutions of theoretical computer science as a field.
Key claims
TCSExam mixes theorem proving, solver engineering, constructions and open-conjecture work behind kernel or task-checker verification.
Qualification: The evaluation produced checked solver outputs, constructions and special-case bounds, not resolutions of theoretical computer science as a field.
Evidence: source-2026-11-07-007
Sources
- arXiv preprint 2611.07962, original v1arXiv · primary research
Corrections
: Publication note: This November 7 edition was assembled before its scheduled 9:00 AM America/Detroit reveal but is being published late after recovery from a local build interruption. Its original reporting, image-generation times and scheduled publishAt are preserved.