[ACL 2025] AgentDropoutV2: The Test-Time "Firewall" for Multi-Agent Reasoning
AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning
AgentDropoutV2 is a test-time "rectify-or-reject" pruning framework for Multi-Agent Systems (MAS). It employs a retrieval-augmented rectifier to iteratively correct agent outputs using a failure-driven indicator pool, achieving a 6.3% average accuracy gain across diverse math benchmarks.
TL;DR
AgentDropoutV2 is a plug-and-play framework that stops the "toxic ripple effect" in Multi-Agent Systems (MAS). By intercepting agent communications and using an offline-mined Indicator Pool to guide iterative corrections, it significantly boosts reasoning accuracy in Math (+6.3%) and Coding without requiring any model retraining.
Background: The "Error Propagation" Nightmare
In a typical MAS (like AutoGen or MetaGPT), agents talk to each other to solve complex problems. However, if Agent A hallucinating a small calculation, Agent B builds on that mistake, and by the time it reaches the final Deciding Agent, the logic is unsalvageable.
Current solutions are either Static (manually designing rigid workflows) or Internalized (fine-tuning the LLM). Neither handles the unpredictable errors that happen at inference time. AgentDropoutV2 introduces a third way: Active Test-Time Intervention.
Methodology: Rectify-or-Reject
The core innovation lies in the tri-state gating mechanism and the Failure-Driven Indicator Pool.
1. The Indicator Pool (Institutional Memory)
The authors don't just ask the agent to "be better." They mined thousands of failed trajectories to create a library of specific pitfalls (e.g., "Conflating Integers with Positive Integers" or "Square Root Non-negativity").
2. The Workflow
- Intercept: Every agent output is paused.
- Retrieve: The system extracts keywords from the agent's logic and retrieves the Top-K relevant "Error Indicators" from the pool.
- Rectify: A Rectifier Model checks the output against these indicators. If it fails, it provides specific diagnostic feedback for the agent to try again (up to 3 times).
- Reject/Prune: If the agent still fails after 3 tries, the output is discarded (pruned) to prevent downstream contamination.
Figure: The pipeline shows how failure-driven mining creates the knowledge base used for test-time intervention.
Experiments: Breaking the AIME Barrier
The framework was tested on grueling math sets like AIME 24/25 and OlympiadBench.
- Math Average: Jumped from 48.95% (Baseline) to 55.25%.
- Task Adaptivity: On easy tasks (GSM8K), most outputs pass on the 1st try. On hard tasks (AIME), the rejection rate exceeds 60%, showing the system knows when to stay silent rather than provide wrong answers.
- Portability: Indicators mined by a large model (Qwen-8B) successfully improved a smaller model (Qwen-4B), suggesting the "logic of failure" is universal across scales.
Table: Significant gains across 9 mathematical benchmarks.
Critical Analysis: Why This Matters
The most striking insight from AgentDropoutV2 is the Correlation as Evaluator. Because the system tracks how many iterations or "rejections" occur, the pruning rate itself becomes a proxy for task difficulty.
Limitations:
- The "Global Fallback" (resetting the whole system if too many agents are pruned) ensures safety but can increase token costs.
- Performance is bottlenecked by the quality of the Rectifier Model. If the "judge" isn't smarter than the "student," the feedback loop breaks.
Conclusion
AgentDropoutV2 treats MAS stability like a network security problem. By implementing a "Rectify-or-Reject" firewall, it transforms MAS from a fragile chain of agents into a robust, self-correcting reasoning engine. It's a vital step toward making production-level agents reliable enough for mission-critical logic tasks.