How does delegation security actually stop repeated mistakes?
Delegation security works by tracking who authorized what, and checking each new action against the accumulated history of the session—not just the initial permissions. The Agentic Principal Chain (APC) [1] does this with six authorization checks, carrying forward and restricting delegated scope and budgets. This means if an agent tries to combine two individually allowed actions into a prohibited outcome, the system catches it because it remembers what came before.
The proof is in the numbers: in tests across 3,154 instances, APC blocked all 544 data-stealing cases in the InjecAgent benchmark and reduced exfiltration in AgentDojo from 75-100% down to 0% across all four domains [1]. Intent binding—making sure the agent sticks to the task—cut destruction from 38.6% to 4.0% and manipulation from 90.5% to 12.1% [1]. For a typical user, this means the system is far less likely to leak your data or take harmful actions, even when an attacker tries to inject malicious instructions.
What's the catch? Does security slow things down or hurt performance?
The main trade-off is a small drop in task utility. In the AgentDojo tests, utility (how well the agent completes its intended task) was 8.6 and 13.9 percentage points lower in the two settings when security was enforced [1]. That's a real cost, but it's modest compared to the security benefit—and the latency overhead is tiny: 0.24 milliseconds at the 99th percentile on an idle host [1], so you won't notice a slowdown.
Another caveat: the security guarantees are proven only under specific conditions. The paper notes that 'Composition Soundness'—the guarantee that prohibited combinations are blocked—holds only for a complete restriction set and serialized admission [1]. In plain terms, if you don't define all the restrictions upfront, or if actions can happen in parallel, the protection may not be airtight. Also, the evaluation used a 'compromised-model' setup where the attack call was inserted after a legitimate tool call, which simulates a worst-case scenario but may not cover every real-world attack path.
Why do mistakes still happen even with guardrails?
Even with delegation security, mistakes can recur because safety constraints tend to 'drift'—they get lost, distorted, or weakened as they pass through memory, delegation, communication, and tool use [3]. The paper on constraint drift argues that prompts and guardrails are necessary but insufficient; constraints must be maintained as explicit execution state, not just asserted at the start [3]. This aligns with the APC approach, which carries forward and restricts delegated scope at each step [1].
The step-by-step verification research [4] reinforces this: process supervision—checking each intermediate step—significantly outperforms outcome supervision (checking only the final result) for training models to solve complex math problems. The process-supervised model solved 78% of problems from a representative subset of the MATH test set [4]. While that's about math reasoning, the principle transfers: checking each step, rather than just the final output, is key to catching errors before they compound. So, delegation security helps, but it must be paired with continuous monitoring of constraints to truly avoid repeated mistakes.
About These Sources
This answer is built on 4 studies (1 peer-reviewed, 3 preprints) — published from 2022 to 2026, 2 from 2024 or later, 1 in Q1 journals, collectively cited 63 times — selected as the most relevant from 4 studies that passed quality screening, drawn from 39 papers retrieved from a database of over 500 million.
Sources used in this answer
Bounded Agents: Delegation Security for Multi-Agent AI Systems
The Agentic Principal Chain (APC) reduced data-stealing attacks to 0% across 544 InjecAgent cases and cut AgentDojo exfiltration from 75-100% to 0%, with a utility cost of 8.6-13.9 percentage points and 0.24 ms latency at the 99th percentile.
Security-preserving multi-agent coordination for complex temporal logic tasks
Proposed a coordination algorithm that synthesizes optimal plans for multi-agent teams to achieve linear temporal logic tasks while preserving each agent's security against passive intruders, demonstrated in real-world experiments.
Safe Multi-Agent Behavior Must Be Maintained, Not Merely Asserted: Constraint Drift in LLM-Based Multi-Agent Systems
Argues that safety-critical constraints in LLM-based multi-agent systems often 'drift'—lose strength over time—and proposes Constraint State Governance to maintain them as explicit execution state, rather than merely asserting them.
Let's Verify Step by Step
Process supervision (checking each intermediate step) significantly outperformed outcome supervision for training models on the MATH dataset, with the process-supervised model solving 78% of problems from a representative subset.