benchmarks evals
Foundation Forecasters Missed the Easy Patterns
SimpleTimeBench found zero-shot failures on monotonic trends, periodic signals and leading indicators.
Summary
SimpleTimeBench found zero-shot failures on monotonic trends, periodic signals and leading indicators.
SimpleTimeBench treats basic temporal relationships as unit tests that a broad forecasting model should nearly solve. Chronos-2, Moirai and Toto frequently produced suboptimal zero-shot forecasts on simple univariate and multivariate patterns, and fine-tuning Chronos-2 for some tasks degraded others. The authors also report underuse of leading indicators in real sensor forecasting, arguing that benchmark scale can hide missing inductive biases for elementary temporal logic.
Why it matters
SimpleTimeBench found zero-shot failures on monotonic trends, periodic signals and leading indicators.
Limits and context
No additional limitation was separately recorded.
Key claims
SimpleTimeBench found zero-shot failures on monotonic trends, periodic signals and leading indicators.
Evidence: source-2026-10-03-014
Sources
- arXiv preprint 2610.02058arXiv · primary research
Corrections
No corrections have been recorded for this story.