research
The Gold Standard Had Missed the Hospital’s Own Names
Institution-specific prompting found locally meaningful protected-health identifiers and exposed 227 omissions in the reference annotations.
Summary
Institution-specific prompting found locally meaningful protected-health identifiers and exposed 227 omissions in the reference annotations.
Across 100 pediatric-oncology notes containing 5,322 annotated PHI spans, the best prompted LLM exceeded two purpose-built de-identification systems in the authors’ comparison. Naming institution-specific categories recovered 48 of 61 misses, and re-annotation confirmed 227 additional PHI spans surfaced by model outputs. No tested multi-agent setup beat the calibrated single pass, and the result belongs to one hospital dataset rather than a universal de-identification guarantee.
Why it matters
Institution-specific prompting found locally meaningful protected-health identifiers and exposed 227 omissions in the reference annotations.
Limits and context
No additional limitation was separately recorded.
Key claims
Institution-specific prompting found locally meaningful protected-health identifiers and exposed 227 omissions in the reference annotations.
Evidence: source-2026-08-19-004
Sources
- arXiv preprint 2608.17051arXiv · primary research
Corrections
No corrections have been recorded for this story.