developer tools
The Right Tool Also Had to Be Working
RATED combines semantic matching with feedback about whether an agent’s tools actually execute.
Summary
RATED combines semantic matching with feedback about whether an agent’s tools actually execute.
Tool retrieval often asks which API matches a request while ignoring whether that API is available. RATED adds a small contextual-bandit reranker that learns from execution feedback without retraining the main retriever. Its EcoAgent benchmark includes 5,172 tools and more than 210,000 query-tool pairs. In the authors’ severe-outage evaluation, fault selection fell from 86.2% to 2.9%; they report less than one millisecond of added latency over encode-only inference. These are benchmark results under specified outage conditions, but they make operational availability part of the selection problem.
Why it matters
RATED combines semantic matching with feedback about whether an agent’s tools actually execute.
Limits and context
- In the authors’ severe-outage evaluation, fault selection fell from 86.2% to 2.9%; they report less than one millisecond of added latency over encode-only inference.
Key claims
RATED combines semantic matching with feedback about whether an agent’s tools actually execute.
Qualification: In the authors’ severe-outage evaluation, fault selection fell from 86.2% to 2.9%; they report less than one millisecond of added latency over encode-only inference.
Evidence: source-2026-10-12-004
Sources
- arXiv preprint 2610.13889, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: The October 12, 2026 edition was prepared and published late on October 12 after the scheduled newsroom task could not start. The edition retains its October 12 date and 9:00 AM America/Detroit reveal time. Reports use sources available before that cutoff; actual preparation and image-generation times are preserved in the provenance records.