research
The Disaster Model Knew the Disaster Too Well
Holding out an entire event exposes shortcuts hidden by random splits of satellite-image tiles.
Summary
Holding out an entire event exposes shortcuts hidden by random splits of satellite-image tiles.
Researchers tested TerraMind, Prithvi-EO-2.0, DOFA and DINOv3 embeddings on two disaster-damage datasets. Mean balanced accuracy fell sharply when an entire disaster was withheld, compared with random image-tile splits. Pre-event images and even annotated building counts remained predictive, suggesting that scene context can stand in for transferable damage evidence. Training-only feature transformations improved generalization in the experiments. The finding concerns the evaluated datasets and embeddings. For disaster mapping, it strengthens the argument that a model should be tested on unfamiliar events before its familiar-event accuracy is treated as operational readiness.
Why it matters
Holding out an entire event exposes shortcuts hidden by random splits of satellite-image tiles.
Limits and context
- Training-only feature transformations improved generalization in the experiments.
Key claims
Holding out an entire event exposes shortcuts hidden by random splits of satellite-image tiles.
Qualification: Training-only feature transformations improved generalization in the experiments.
Evidence: source-2026-10-13-006
Sources
- arXiv preprint 2610.13766, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: The October 13, 2026 edition was prepared and published late on October 13 after the scheduled newsroom task could not start. It retains its October 13 edition date and 9:00 AM America/Detroit reveal time. Reports use evidence available before that cutoff; actual preparation and image-generation times are preserved in the provenance records.