How do agents know what actually worked?
The first hurdle is feedback reliability: an agent can only evolve if it can tell a good change from a bad one. The survey on self-evolving coding agents [2] highlights that executable feedback—like running tests—is a natural signal for coding agents, but it's not always trustworthy. Tests can be flaky, incomplete, or even wrong, and the survey explicitly warns about benchmark overfitting, where an agent tunes itself to pass a specific test set but fails on new, real-world problems. So before agents can evolve outside demos, they need feedback mechanisms that distinguish genuine improvement from lucky test passes.
The SEW framework [4] shows one way to address this: it automatically designs and optimizes multi-agent workflows, and on the challenging LiveCodeBench benchmark, it improved performance by up to 33% compared to using the base LLM alone. That's a big jump, but it also underscores the problem—the improvement is measured on benchmarks, which are exactly the kind of controlled settings where overfitting is a risk. The authors themselves note that the approach needs to adapt to different coding problems, implying that current feedback signals are still too narrow to generalize.
How do agents remember without forgetting?
A second major problem is memory: agents can't just keep piling up text experiences, because that eventually exhausts the model's context window and slows retrieval. The self-consolidation paper [1] directly tackles this by proposing a mechanism that distills non-parametric textual experience into compact learnable parameters, allowing the agent to internalize historical experience into its latent space. This is a crucial step because it addresses the scalability issue—without such consolidation, an agent's memory becomes a bottleneck, and the paper notes that simply accumulating textual experiences introduces noise and exhausts the context window.
The same paper also introduces a contrastive reflection strategy that explicitly summarizes error-prone patterns, rather than just learning from successes. This is important because failed attempts carry pedagogical value—they teach the agent what to avoid. The survey [2] similarly mentions that coding trajectories are a rich source of experience, but the challenge is to extract reusable insights without drowning in noise. So, before agents can evolve in the wild, they need memory systems that are both compact and selective, not just bigger.
Can agents change their own goals without breaking things?
The third challenge is safety and stability: when an agent evolves its own goals and code, it risks losing the behaviors that already work. The BDI-LLM architecture paper [3] shows that agents can autonomously discover new goals and generate executable behaviors from minimal prior knowledge, but the authors explicitly note 'current limits' in terms of behavioral inheritance and stability. In other words, the agent might evolve in a way that forgets or breaks previously learned skills—a serious problem for real-world deployment where reliability is paramount.
The survey [2] echoes this concern, listing 'reversibility' and 'system complexity' as open challenges. If an agent makes a bad change, can it roll back? How do you ensure that a self-modifying system remains safe and predictable? These are not just technical details; they are fundamental to whether self-evolving agents can be trusted outside controlled demos. The papers collectively suggest that while evolution is feasible, ensuring that it is stable and reversible is still unsolved.
About These Sources
This answer is built on 4 studies (1 peer-reviewed, 3 preprints) — published from 2025 to 2026, 4 from 2024 or later — selected as the most relevant from 4 studies that passed quality screening, drawn from 21 papers retrieved from a database of over 500 million.
Sources used in this answer
Self-Consolidation for Self-Evolving Agents
Proposes a self-evolving framework with contrastive reflection and self-consolidation to distill textual experience into compact learnable parameters, addressing context-window exhaustion and noise from accumulating experiences.
Self-Evolving Coding Agents
Survey of self-evolving coding agents that identifies challenges including feedback reliability, benchmark overfitting, reversibility, system complexity, safety, cost, and generalization, while noting that executable feedback and repository-level context make software engineering a natural domain for self-evolution.
Self-Evolving Software Agents
Introduces a BDI-LLM architecture for self-evolving software agents that can autonomously discover new goals and generate executable behaviors, but notes current limits in behavioral inheritance and stability based on evaluation in a dynamic multi-agent environment.
SEW: Self-Evolving Agentic Workflows for Automated Code Generation
Presents SEW, a self-evolving workflow framework that automatically designs and optimizes multi-agent workflows, achieving up to 33% improvement on LiveCodeBench compared to using the backbone LLM alone.