business enterprise
The Cheap Classifier Knows When to Hand Off
On one 47,837-ticket corpus, calibrated TF-IDF routing cut the illustrative test cost 38.1 percent and wrong routes from 15.3 to 3.5 percent.
Summary
On one 47,837-ticket corpus, calibrated TF-IDF routing cut the illustrative test cost 38.1 percent and wrong routes from 15.3 to 3.5 percent.
Misrouting an IT ticket may cost more than asking a person to triage it. Under an illustrative cost model where a wrong route costs five times a human deferral, a calibrated TF-IDF logistic-regression system reduced test cost from 0.763 to 0.472 and cut wrong routes from 15.3 to 3.5 percent. The strongest prompted large-language-model configuration reached 0.477 accuracy on a 1,000-ticket subset, versus 0.879 for fine-tuned DistilBERT. Several prompted systems deferred everything at the break-even threshold. The study is a readiness framework for one dataset and cost model, not a universal verdict on language models or support operations.
Why it matters
On one 47,837-ticket corpus, calibrated TF-IDF routing cut the illustrative test cost 38.1 percent and wrong routes from 15.3 to 3.5 percent.
Limits and context
- The study is a readiness framework for one dataset and cost model, not a universal verdict on language models or support operations.
Key claims
On one 47,837-ticket corpus, calibrated TF-IDF routing cut the illustrative test cost 38.1 percent and wrong routes from 15.3 to 3.5 percent.
Qualification: The study is a readiness framework for one dataset and cost model, not a universal verdict on language models or support operations.
Evidence: source-2026-10-23-009
Sources
- arXiv preprint 2610.31657, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.