benchmarks evals
The Recommender Won by Breaking Ties in Its Favor
A tie-aware reanalysis found that several reported group-recommendation gains shrink or reverse under fairer scoring.
Summary
A tie-aware reanalysis found that several reported group-recommendation gains shrink or reverse under fairer scoring.
An extra sigmoid before a common ranking objective can compress top scores until many items tie. Deterministic evaluation then lets an implementation detail decide hit-rate and NDCG results. Re-evaluating representative methods on two benchmarks with the exact expectation under random tie-breaking, the authors found that many gains narrowed and some model rankings changed. Temperature scaling retained some optimization benefit without the same tie inflation.
Why it matters
A tie-aware reanalysis found that several reported group-recommendation gains shrink or reverse under fairer scoring.
Limits and context
No additional limitation was separately recorded.
Key claims
A tie-aware reanalysis found that several reported group-recommendation gains shrink or reverse under fairer scoring.
Evidence: source-2026-08-12-007
Sources
- arXiv preprint 2608.11190arXiv · primary research
Corrections
No corrections have been recorded for this story.