The Evolution of Autonomy: A Survey of Self-Improving Agentic Systems

    ## Executive Summary
    **TL;DR**: We are transitioning from static LLM pipelines to **Self-Improving Agentic Systems** that autonomously refine their own parameters and operational logic. This paper provides the first unified taxonomy—the "Core-and-Scaffold" framework—to categorize how agents learn from experience, optimize their toolsets, and even rewrite their own source code.

    **Academic Positioning**: This survey is a foundational mapping effort. It bridges the gap between 1990s meta-learning theories (like Schmidhuber’s "Learning to Think") and modern LLM-based agents, positioning self-improvement as the ultimate path toward AGI.

    ## Problem & Motivation: The Static Bottleneck
    Traditional AI agents suffer from a "frozen intelligence" problem. Once a model is trained and its prompts are written, its capability ceiling is fixed. If it fails a task today, it will likely fail it tomorrow.
    
    The authors identify that the real bottleneck isn't just the **Foundation Model (FM)** weights, but the **Scaffolding**—the external memory, tools, and prompts that wrap the model. Prior work treated these as separate engineering problems; this survey argues they are two halves of the same self-improvement loop.

    ## Methodology: The Core-and-Scaffold Framework
    The authors formalize an agent as $\mathcal{A}_t = (	heta_t, \Sigma_t)$. 
    - **$	heta$ (The Core)**: The neural parameters. Updating this is "Slow Consolidation."
    - **$\Sigma$ (The Scaffold)**: The prompts, memory, and tools. Updating this is "Fast Adaptation."

    ![Overview of self-improvement paradigms](https://cdn.atominnolab.com/wisdoc/images/20260717-ee562b65-8d3d-4473-82ce-cb23df2b08c7/page_000_block_006.png)

    ### Pathway 1: Foundation Model Improvement
    Here, the agent acts as its own teacher. It generates **Intrinsic Demonstrations** (synthetic data) or uses **Intrinsic Evaluative Feedback** (acting as its own judge) to fine-tune its weights. This internalizes skills into the model’s "intuition."

    ### Pathway 2: Scaffolding Improvement
    This is essentially "Self-Referential Engineering." 
    1. **Prompt Optimization**: Using "Textual Gradients" to refine instructions.
    2. **Memory Evolution**: Moving from flat logs to hierarchical graphs that "reason" about what to remember.
    3. **Tool Creation**: Agents that write their own Python scripts to solve new math problems and store them as permanent tools.

    ## Experiments & Results: Proving the Loop
    The survey maps these techniques across several high-stakes domains:
    - **Software Engineering**: Systems like the *Darwin-Gödel Machine* can solve GitHub issues by iteratively patching their own logic.
    - **Web Navigation**: *WebRL* uses a self-evolving curriculum to navigate complex, dynamic sites, achieving SOTA results with minimal human supervision.

    ![Timeline and Taxonomy](https://cdn.atominnolab.com/wisdoc/images/20260717-ee562b65-8d3d-4473-82ce-cb23df2b08c7/page_006_block_001.png)

    The "Capability Trap" is a key finding: agents can easily over-optimize to a specific "Judge" (LLM-as-a-judge), leading to **Reward Hacking**. The paper emphasizes that for self-improvement to be real, it must show **Transferability** to tasks the agent has never seen.

    ## Critical Analysis & Conclusion
    **Takeaway**: We are moving away from "prompt engineering" toward "evolutionary engineering." The future of AI isn't a better model, but a better **evolutionary mechanism**.

    **Limitations**:
    - **Security**: A self-improving agent is essentially "untrusted code." If an agent can rewrite its scaffold, it can bypass safety filters.
    - **Model Collapse**: Recursive training on synthetic data risks "intellectual incest," where the agent grows more confident but less accurate.

    **Future Perspective**: The authors suggest a shift toward **Neural Runtimes**—where the boundary between the "code" of the agent and the "weights" of the model completely disappears.