EVOTOOL: "Surgical" Policy Optimization for LLM Agents via Blame-Aware Evolution
EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection
EVOTOOL is a gradient-free, self-evolving framework designed to optimize tool-use policies in LLM agents by decomposing them into four functional modules: Planner, Selector, Caller, and Synthesizer. It achieves state-of-the-art performance, outperforming baselines by over 5 points on benchmarks like ToolBench and -Bench using both GPT-4.1 and Qwen3-8B.
TL;DR
Optimizing LLM agents is notoriously difficult because a failure at step 10 might actually be caused by a bad plan at step 1. EVOTOOL solves this by decomposing the agent into a modular pipeline (Planner, Selector, Caller, Synthesizer) and using a "Blamer" LLM to pinpoint exactly which module failed. By evolving only the "guilty" module through natural language feedback, it reaches SOTA performance on complex benchmarks with 5-7% gains over traditional monolithic prompt optimizers.
The Credit Assignment Crisis in Agents
The current paradigm for building agents often relies on a single, massive prompt. This creates a "Monolithic Policy" problem: if you try to fix the agent's tool-calling syntax, you might accidentally break its high-level reasoning capabilities (catastrophic forgetting/interference).
Conversely, "Single-aspect" optimizers target only one part (e.g., just the planner), ignoring the reality that tool-use is a relay race. If the Selector picks the wrong tool, the Caller cannot possibly succeed, no matter how well-optimized its syntax is. This credit assignment dilemma—knowing who to blame for a failure in a long-horizon trajectory—is the primary bottleneck in agentic R&D.
Methodology: The Self-Evolving Loop
EVOTOOL treats agent optimization as a biological evolution process, but with a highly logical "selection" and "mutation" phase.
1. Modular Architecture
Instead of one prompt, the policy is split into:
- Planner: Decomposes subgoals.
- Selector: Chooses the tool.
- Caller: Formats the API arguments.
- Synthesizer: Merges results into a final answer.
2. Trajectory-Grounded Blame Attribution
When a trajectory fails, a Blamer LLM analyzes the trace. It looks for specific "diagnostic events" (e.g., did the API return a 400 error? Did the synthesizer hallucinate facts?). It assigns a Blame Score to each module. The module with the highest score is targeted for mutation.
3. Targeted Mutation & Diversity Selection
Instead of random changes, a Mutator LLM generates natural language feedback based on the failure. It revises only the blamed module's prompt. To avoid "mode collapse" (where the agent becomes a specialist in one task but fails others), EVOTOOL uses Diversity-Aware Population Selection. It keeps a candidate in the population if it is the "best" for at least one specific instance, ensuring a "committee of experts" approach to evolution.
Figure 1: The EVOTOOL workflow: collection, localization (blame), targeted mutation, and diversity-preserved selection.
Experimental Results: SOTA Efficiency
EVOTOOL was tested against heavyweights like OPRO, PromptBreeder, and DRAFT across four benchmarks (ToolBench, RestBench, -Bench, and BFCL).
Performance Dominance
On GPT-4.1, EVOTOOL achieved an overall average of 70.6, a massive jump from the 60.6 achieved by standard ReAct prompting. Even more impressive is its performance on Qwen3-8B, where it outperformed the next best method by 5.2 points, proving it can "teach" smaller models to use tools as effectively as giants.
Token Efficiency
Because EVOTOOL only mutates one module at a time and targets the "root cause" of failures, it is significantly more token-efficient. It reaches higher performance levels much faster than monolithic methods that attempt to optimize the entire prompt at once.
Figure 2: Performance vs log token cost. EVOTOOL (red line) consistently stays in the upper-left quadrant (high performance, low cost).
Deep Insights: Why it Works
A key finding in the ablation studies was that Random Mutation is destructive. Simply changing the prompt without knowing what to change reduces performance by over 9 points.
The "Blamer" effectively creates a local gradient in a discrete search space. By identifying that the Planner is the bottleneck in long-horizon tasks (like -Bench) versus the Selector being the bottleneck in large API tasks (like ToolBench), EVOTOOL adaptively shifts its optimization energy where it's needed most.
Qualitative Evolution
In the early stages, the Planner prompts are generic. By the 8th iteration, the evolved prompt includes specific "interface contracts" like:
- "Prioritize early acquisition of required IDs."
- "Atomic step-to-capability mapping."
- "Standardized stateful variable storage."
Conclusion & Future Outlook
EVOTOOL demonstrates that modularity is the antidote to complexity in LLM agents. By treating the agent as a collection of specialized components rather than a single black box, we can apply "surgical" enhancements that are both interpretable and highly effective.
Limitations: The evolutionary process still requires multiple iterations, which adds latency to the development cycle. Future work might involve "on-the-fly" blame attribution to correct behaviors in real-time without needing a full offline evolution loop.
Takeaway for Practitioners: Stop optimizing your 2000-word system prompt as a single block. Break it into modules, identify the failure points of each, and use LLM-based feedback to refine them individually.