MSCE: Turning Noisy Agent Memories into Precision Skills
From Memory to Skills: Evidence-Grounded Co-Evolution Governance for Long-Horizon LLM Agents
The paper introduces MSCE (Memory-Skill Co-Evolution), a training-free framework for long-horizon LLM agents. It transforms passive agent experiences into a hierarchical memory structure—grounded traces (L1), procedural policies (L2), and environmental cognition (L3)—and crystallizes high-utility policies into executable Skills.
TL;DR
LLM agents often suffer from "context bloat"—re-reading old logs without actually learning new tricks. MSCE (Memory-Skill Co-Evolution) changes the game by introducing a governance layer that distills raw interaction traces into a hierarchy of procedural policies and environmental facts. By crystallizing these into "Callable Skills," agents become more efficient (up to 50% fewer turns) and significantly more successful in long-horizon tasks like software engineering.
The Problem: The "Passive Context" Trap
Most current LLM agents (like those using simple RAG or basic reflection) store experience as a flat stack of text. When an agent faces a task it has solved before—like installing a specific library on a niche OS—it retrieves the old logs and re-reasons through the failure and success all over again.
This is problematic because:
- High Noise: Raw trajectories contain failed attempts and redundant steps.
- Credit Assignment Junk: Sparse terminal feedback (just "Success" or "Failure") makes it hard to know which specific step was the "ah-ha!" moment.
- Brittle Reusability: Fact-based memory doesn't translate to action-oriented skills.
Methodology: The Three-Tier Evolution
The core innovation of MSCE is its structured evolution path. It doesn't just "remember"; it "governs."
1. The Memory Hierarchy
MSCE organizes knowledge into three distinct layers:
- L1 (Trace Memory): The "Evidence" layer. It stores grounded decision units (actions, observations, and reflections).
- L2 (Policy Memory): The "Abstaction" layer. It induces recurring procedural patterns from L1. If an agent fixes a dependency bug three times, L2 creates a "Dependency Resolution Policy."
- L3 (Environmental Cognition): The "World Model" layer. It consolidates declarative facts (e.g., "This server uses Alpine Linux, which lacks glibc") to provide context for future decisions.
2. Reflection-Weighted Value Backfilling
How do we know which memory is worth keeping? MSCE uses Dual-Signal Feedback. It takes the final reward (Global) and weights it against the agent's internal self-reflection (Local). This "Reflection Weight" () ensures that steps that are both successful and locally interpretable get high value-marks, guiding which policies eventually become "Skills."
Figure 1: Overview of MSCE with governed memory and skill crystallization.
From Policy to Skill: The "Crystallization" Gate
A policy in L2 isn't automatically a skill. MSCE acts as a strict editor through two gates:
- Positive Gain: Does this policy actually improve performance compared to not having it?
- Stability: Has the procedure remained consistent over recent attempts?
If it passes, the policy is Crystallized into a Skill—a structured object with a name, parameters, preconditions, and verification rules. This turns the LLM from a "reasoner-from-scratch" into a "library-caller."
Experimental Triumphs
The results on EvoAgentBench demonstrate a clear "Learning by Using" effect.
| Domain | SOTA Baseline (Pass@1) | MSCE (Pass@1) | Improvement |
|---|---|---|---|
| Software Engineering | 38.64 | 53.85 | +15.39 pts |
| Math Reasoning | 43.00 | 47.00 | +4.00 pts |
| Info Retrieval | 21.54 | 26.15 | +4.61 pts |
Even more impressive is the Cost-Efficiency. In the Code Implementation domain, MSCE achieved the same success rate as the top baseline but slashed the turn count from 3.9 to 2.0.
Figure 2: Performance improvement over time. As experience scales (p0 to p100), success increases while cost (after an initial learning hump) decreases.
Critical Insight: Why This Matters
MSCE proves that Governance is as important as Generation. By inserting a structured L1-L2-L3 hierarchy between raw data and agent action, MSCE creates an "Operating System" for intelligence.
Limitations: The framework currently relies heavily on prompted "Judge" models for value calculation, which adds latency and API costs. Furthermore, the credit assignment is heuristic rather than truly causal.
Future Outlook: MSCE sets the stage for agents that can transfer skills across disparate domains (e.g., learning debugging patterns in Python and applying them to Shell scripts) without needing to be re-trained from the ground up.
Conclusion
MSCE represents a shift from Big Context to Smart Abstraction. It shows that for agents to operate over long horizons, they don't need to remember everything—they need to turn what they remember into executable expertise.