Workflow · Production

One Line of Code, 41% Better Memory: When One AI Agent Optimizes Another

The problem

Coding agents lose all context between sessions, and Lerim's memory extraction and deduplication quality was uncertain — there was room to improve but no clarity on which parts of the system needed it.

First attempt

The initial evaluation harness measured the wrong thing — rewarding recall without penalizing over-extraction — so the memory store accumulated low-value entries despite high eval scores.

Workflow diagram · grounded in source
1
Setup optimization harness
trigger
“I pointed Claude Code (Opus 4.6) at Lerim's codebase, gave it an eval harness and a golden dataset, and told it to optimize”
2
Claude Code reads and modifies
ai_action
“An AI agent reading another AI system's code, understanding the design tradeoff, and making the right call”
3
Evaluator scores pipeline
validation
“A composite score (extraction quality, search relevance, dedup accuracy, maintain precision) measured against 15 golden test cases. Claude Code cannot touch the evaluator”
4
Keep or revert change
routing
“keeping what works, reverting what doesn't”
5
End-to-end lifecycle validation
validation
“I ran the full lifecycle eval on both the original and optimized code to verify the gains hold up end-to-end”
6
Failure log feeds next round
feedback_loop
“Seven experiments failed. Each failure narrowed the search space — "don't change tool descriptions in isolation" is knowledge that prevents future mistakes”
Reported outcome

Round 1 achieved a 41% improvement in composite quality score, with dedup accuracy rising from 0.28 to 0.72 and maintain improving by 29% as a cascade effect.
Round 2 added a further 3.4% extraction quality improvement by teaching the LLM explicit quality criteria.

Reported metrics
composite quality score improvement (Round 1)41%
Dedup accuracy improvementfrom 0.28 to 0.72
Maintain improvement (end-to-end cascade)29%
extraction quality improvement (Round 2)+3.4%
Show all 10 reported metrics
composite quality score improvement (Round 1)41%
dedup accuracy improvementfrom 0.28 to 0.72
maintain improvement (end-to-end cascade)29%
extraction quality improvement (Round 2)+3.4%
Round 1 experiments run14
Round 1 experiments kept vs discardedSeven were kept, seven were discarded
Round 2 experiments run10
golden dataset size (Round 2)327
exp021 quality criteria gain+2.2%
exp022 body structure gain+0.7%
Reported stack
Claude CodeLerimDSPyPydanticMiniMax M2.5AutoResearch
Source
https://kargarisaac.medium.com/one-line-of-code-41-better-memory-when-one-ai-agent-optimizes-another-da2396bc501b
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Round 1 achieved a 41% improvement in composite quality score, with dedup accuracy rising from 0.28 to 0.72 and maintain improving by 29% as a cascade effect.

What tools did this team use?

Claude Code, Lerim, DSPy, Pydantic, MiniMax M2.5, AutoResearch.

What results were reported?

composite quality score improvement (Round 1): 41%; Dedup accuracy improvement: from 0.28 to 0.72; Maintain improvement (end-to-end cascade): 29%; extraction quality improvement (Round 2): +3.4% (source-reported, not independently verified).

What failed first in this deployment?

The initial evaluation harness measured the wrong thing — rewarding recall without penalizing over-extraction — so the memory store accumulated low-value entries despite high eval scores.

How is this workflow AI workflow structured?

Setup optimization harness → Claude Code reads and modifies → Evaluator scores pipeline → Keep or revert change → End-to-end lifecycle validation → Failure log feeds next round.