TL;DR

Even the most capable LLMs (like Claude 3.5 Sonnet) fail to collaborate on code when the "rules of the game" are slightly blurry. This paper uncovers a massive Specification Gap: when two agents implement parts of the same Python class without a shared data-structure contract, their integration success plummets from 58% to 25%. The fix isn't smarter conflict detection—it's richer, more explicit specifications.

Positioning: This is a critical "reality check" for the move from single-turn code generation to complex, multi-agent software engineering (MAS) workflows.

Problem & Motivation: The Silent Failure of Autonomy

In traditional software engineering, we use "Information Hiding" and "Design by Contract" to make sure modules fit together. But LLM agents are stochastic. If Agent A decides to store user data in a list and Agent B expects a dict, both might write "correct" code that crashes the moment they are merged.

The authors identify that this Partial Knowledge problem creates errors distinct from logic bugs—it creates structural incompatibility. They set out to measure exactly how much detail an LLM needs to "stay in sync" with a partner it cannot talk to.

Methodology: Stress-Testing Coordination

The researchers created AmbigClass, a subset of ClassEval, and tested agents across four levels of specification:

• L0 (Full): Docstrings, doctests, and explicit data-structure mentions.
• L1-L2: Gradually stripping detail.
• L3 (Bare): Only method signatures.

To force a "worst-case scenario," they gave agents opposing biases: Agent A was told to love Lists; Agent B was told to love Dictionaries.

Figure 1: The experimental pipeline. A single agent (top) serves as the ceiling, while split biased agents (bottom) attempt to coordinate via the specification.

Key Insights & Results

1. The Coordination Tax is Real and Expensive

The "Coordination Tax"—the performance lost simply because the task was split—is huge. Even at L0 (the best spec), two agents performed 30 percentage points worse than a single agent implementing the whole class. This gap never disappears, regardless of model capability (tested on Sonnet and Haiku).

2. Diagnosis $

eq$ Cure The authors built a "zero-cost" AST (Abstract Syntax Tree) conflict detector. While it was 97% accurate at flagging when agents disagreed on types (List vs. Dict) at the L3 level, giving these conflict reports to a "Merger Agent" didn't help it fix the code.

3. Specification is the Sufficient Recovery Instrument

In a brilliant $2 imes 2$ factorial experiment, the authors found that if a merger agent is given the Full Specification (L0), it can fix the broken code 89% of the time, even without knowing where the conflicts were.

• Spec Effect: +36.2pp improvement.
• Conflict Report Effect: 0.0pp (literally no help).

Figure 2: Restoring the specification recovers the single-agent ceiling, proving that specifications are the primary coordination mechanism.

Critical Analysis: Why This Matters for AI Engineers

This paper challenges the hype around "self-healing" multi-agent systems that rely on iterative debugging or post-hoc analysis. The core takeaway is Specification-First Orchestration.

If you are building an AI coding assistant:

1. Don't just pass method signatures. Agents need to know the specific internal state (e.g., "self.data is a dict mapping UUID to Int").
2. Coordination vs. Information: The study separates these two, showing that 16pp of the failure is pure "coordination cost" (the difficulty of making shared decisions), while 11pp is "information asymmetry" (not seeing the constructor).
3. AST Monitoring: Use AST-based detectors as a monitoring signal to alert humans that a spec is too vague, rather than using it to try to "auto-fix" the merge.

Limitations & Future Work

The study uses a "bias injection" (forcing List vs. Dict) which is a stylized proxy for real-world stylistic drift. While the authors proved the gap persists even without this bias (the init-visibility experiment), real-world conflicts might be subtler and "squishier" than a simple Type error. Future research should explore if iterative negotiation (letting Agent A and B chat before coding) can close the gap better than a static specification.

Conclusion

The path to reliable multi-agent code generation isn't just "better models"—it's better engineering discipline. LLM agents, like human developers, require clear contracts to work together. Without them, the specification gap remains an insurmountable wall.