How should researchers explain the limits of delegation security for multi-agent AI to non-experts?

The core trade-off: autonomy versus control

The central tension in multi-agent AI is that giving agents more autonomy to act on your behalf also increases the chance they'll do something you didn't intend. This isn't just a theoretical worry—one paper notes that autonomous agents can make thousands of API calls per hour without human oversight, meaning a single misstep can cascade quickly [1]. The same paper argues that traditional security standards like OAuth 2.0 assume a predictable, single-step client, but agentic systems are stochastic and can be manipulated, so privileges can silently expand [1]. For non-experts, the takeaway is: every time you delegate a task to an AI agent, you're handing over a piece of your authority, and the more steps in the chain, the more chances for something to go wrong.

Another paper frames this as a 'confused-deputy' problem: an agent may follow instructions from an untrusted source (like a malicious email) and inadvertently perform actions on your behalf [5]. This is why security experts talk about 'least privilege'—giving agents only the minimum access they need. But in practice, that's hard to enforce because agents often need broad access to tools and data to be useful. So the trade-off is real: more autonomy means more utility, but also more risk. The studies here agree that this trade-off is unavoidable, and the solution is not to eliminate risk but to manage it with layered defenses [3][5].

What can go wrong: concrete threats non-experts should know

When explaining risks, it helps to name specific failure modes. One paper lists several: unauthorized compliance with non-owner instructions, sensitive information disclosure, identity spoofing, and cross-agent propagation of unsafe practices [3]. In healthcare, these become potential HIPAA violations, but the same risks apply to finance, legal, or any high-stakes domain. Another paper highlights indirect prompt injection—where an attacker hides malicious instructions in external content (like a webpage or document) that an agent reads, causing it to act against your interests [5]. These are not hypothetical; the healthcare paper reports that in a 90-day production deployment, an automated security audit agent found and remediated four HIGH severity issues [3].

A key point for non-experts is that these threats are not just about hackers breaking in—they're about the agent being tricked or making a mistake. For example, an agent might be socially engineered into revealing confidential data, or it might follow a chain of delegation that goes beyond what you authorized. One paper introduces a protocol specifically to track the 'provenance' of human authorization, so you can verify that a terminal action was genuinely approved by a human and not just by a chain of agents [2]. This is like a digital paper trail for AI actions, but it's not yet widely adopted.

What protections exist—and their limits

The good news is that researchers are developing concrete defenses. One approach is to bind each agent's action to a verifiable user intent token, like a digital signature that proves the action was authorized [1]. Another is to use a lightweight cryptographic protocol that records each delegation step in an append-only chain, so you can audit the entire path [2]. In practice, some organizations are already deploying layered defenses: kernel-level isolation (sandboxing), credential proxies that keep secrets away from agents, network egress policies that restrict where agents can send data, and prompt integrity frameworks that label untrusted content [3]. These are not silver bullets, but they reduce the attack surface.

However, the studies also show that these protections have limits. For example, the Agentic JWT protocol adds sub-millisecond overhead, which is fast, but it's a proof-of-concept, not a standard [1]. The HDP protocol is published as an IETF draft, but it's not yet widely implemented [2]. And the healthcare architecture, while effective in one deployment, required significant engineering effort and is not a plug-and-play solution [3]. For non-experts, the message is: security is a moving target, and you should expect that no single tool will make multi-agent AI completely safe. Instead, look for systems that combine multiple layers and that allow you to audit what agents did.

How to talk about residual risk without scaring people

When explaining limits to non-experts, avoid technical jargon and use analogies. For instance, compare delegation to giving a trusted employee a key to your office: you trust them, but you also install cameras and limit what they can access. Similarly, AI agents need credentials, but you can restrict their access and monitor their actions. One paper suggests that current standards are insufficient for multi-agent chains, so you should assume that some risk remains [2]. Another paper emphasizes that security is not a one-time fix but an ongoing process, with open challenges like runtime monitoring and verification [4].

A practical way to frame residual risk is to say: 'We can reduce the chance of unauthorized actions, but we cannot eliminate it entirely. That's why we need to monitor and audit.' The studies here support this: they all point to the need for continuous evaluation and adaptation, not just a one-time setup [4][5]. For example, the survey on trustworthy agentic AI highlights that multi-step trajectories introduce new failure modes, and that we need both outcome and process signals (like constraint violations and trace completeness) to evaluate trustworthiness [4]. So, when explaining limits, emphasize that security is a journey, not a destination, and that the best systems are those that are transparent about their limitations and provide ways to verify what happened.

About These Sources

This answer is built on 5 studies (1 peer-reviewed, 4 preprints) — published from 2025 to 2026, 5 from 2024 or later — selected as the most relevant from 6 studies that passed quality screening, drawn from 48 papers retrieved from a database of over 500 million.

Sources used in this answer

Related research questions