developer tools
The Agent Checked Its Memory Before Believing It
Read-only environment probes lifted one database benchmark’s pass rate from 39% to 73% while cutting task cost roughly in half.
Summary
Read-only environment probes lifted one database benchmark’s pass rate from 39% to 73% while cutting task cost roughly in half.
Persistent agent memory can preserve stale facts and overgeneralized lessons when its curator sees only a completed trajectory. The new environment-probing approach gives an asynchronous curator least-privilege, read-only tools to verify, scope and refresh candidate memories without changing the task agent or granting production write authority. In the authors’ GitHub Copilot harness, CLBench pass rate rose from 39% to 73%, average queries fell from 8.8 to 4.7, and task-agent cost dropped from $3.38 to $1.68. Across six adapted consulting worlds, all 18 memory-versus-baseline reward comparisons were positive and tool calls fell 16% to 75%. These are benchmark results from a production-like harness, not an independent audit of deployed enterprise agents.
Why it matters
Read-only environment probes lifted one database benchmark’s pass rate from 39% to 73% while cutting task cost roughly in half.
Limits and context
- Persistent agent memory can preserve stale facts and overgeneralized lessons when its curator sees only a completed trajectory.
- The new environment-probing approach gives an asynchronous curator least-privilege, read-only tools to verify, scope and refresh candidate memories without changing the task agent or granting production write authority.
- These are benchmark results from a production-like harness, not an independent audit of deployed enterprise agents.
Key claims
Read-only environment probes lifted one database benchmark’s pass rate from 39% to 73% while cutting task cost roughly in half.
Qualification: Persistent agent memory can preserve stale facts and overgeneralized lessons when its curator sees only a completed trajectory.
Evidence: source-2026-09-11-001
Sources
- arXiv preprint 2609.11060arXiv · primary research
Corrections
No corrections have been recorded for this story.