benchmarks evals
A Team Memory Learns When It Should Stay Quiet
DiscernMem tests repairs against current and earlier tasks, then learns the contexts in which each role should apply or avoid a stored rule.
Summary
DiscernMem tests repairs against current and earlier tasks, then learns the contexts in which each role should apply or avoid a stored rule.
A successful multi-agent episode does not reveal whether a stored memory was sound, irrelevant or accidentally helpful. DiscernMem compares executions with one memory against runs without it, diagnoses defects and tests proposed repairs on both current and earlier tasks to avoid destroying previously useful behavior. A second component records each memory's past use and effect, learning a contextual boundary for when a role should apply or avoid it. With Qwen3-4B-Instruct-2507 across four benchmarks, the training-free framework led reported baselines on AutoGen, CAMEL and MacNet by 0.51 to 4.32 average points and improved symbolic-planning results. The evidence belongs to the tested model, systems and tasks rather than every multi-agent memory design.
Why it matters
DiscernMem tests repairs against current and earlier tasks, then learns the contexts in which each role should apply or avoid a stored rule.
Limits and context
- A successful multi-agent episode does not reveal whether a stored memory was sound, irrelevant or accidentally helpful.
Key claims
DiscernMem tests repairs against current and earlier tasks, then learns the contexts in which each role should apply or avoid a stored rule.
Qualification: A successful multi-agent episode does not reveal whether a stored memory was sound, irrelevant or accidentally helpful.
Evidence: source-2026-10-28-010
Sources
- arXiv preprint 2610.36652, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.