benchmarks evals
Deep Tabular Generators Rarely Beat the Trivial Baseline
A preregistered 2,220-run size ladder found no meaningful deep-model win in 23 of 24 dataset-model pairs.
Summary
A preregistered 2,220-run size ladder found no meaningful deep-model win in 23 of 24 dataset-model pairs.
The benchmark compared seven generators across eight datasets subsampled from 200 to 20,000 rows, plus four naturally small clinical datasets. In 23 of 24 dataset-and-deep-model pairs, no deep model beat the best trivial baseline beyond seed noise at any measured size; baselines won 40 of 49 size cells. In 81 percent of cells, even the top-two gap was smaller than seed variation.
Why it matters
A preregistered 2,220-run size ladder found no meaningful deep-model win in 23 of 24 dataset-model pairs.
Limits and context
No additional limitation was separately recorded.
Key claims
A preregistered 2,220-run size ladder found no meaningful deep-model win in 23 of 24 dataset-model pairs.
Evidence: source-2026-10-05-015
Sources
- arXiv preprint 2610.03500arXiv · primary research
Corrections
No corrections have been recorded for this story.