A benchmark that scores the whole attack chain
LLMail-Inject is a public red-teaming challenge built around a simulated LLM email assistant. A participant writes one email; a simulated user asks the assistant to summarize or answer questions about a mailbox; a retriever pulls relevant emails into the model's context; and the model may call a send_email tool. The attacker's goal is to make the assistant call that tool with the wrong arguments — for example, sending a message to an address the user never named.
The key design choice is that success is not judged on whether the model merely repeats injected text. A submission counts only if the attacker's email is retrieved, survives every deployed defense, and produces a tool call with the correct arguments. This is why the paper describes the benchmark as measuring the complete chain of execution rather than a single classification step.
The corpus behind it is large: 839 participants submitted 461,640 attempts over two phases spanning five months, yielding 208,095 unique prompts. Because participants wrote the attacks themselves and knew which defenses were deployed, the dataset captures social-engineering and persuasion styles rather than only explicit strings like "ignore previous instructions".
Why template benchmarks understate the risk
The paper's motivating comparison is a gap between two kinds of measurement. Existing prompt-injection benchmarks typically contain 100 to 1,000 attack examples built from fixed templates. Defenses evaluated on those sets can overfit to narrow patterns, and models show saturating performance — which the authors read as a sign the benchmarks no longer discriminate.
LLMail-Inject changes the denominator. Instead of asking whether a classifier flags a known template, it asks how often a human-crafted email, aimed at a specific sub-level, actually drives an unauthorized tool call. The same models look much weaker under this criterion: Meta SecAlign, which the paper notes was specifically trained against prompt injection and achieves under 5% ASR on prior benchmarks, reaches 32% ASR here; GPT-4o reaches 58% and GPT-5 mini 47%.
These figures come from replaying the collected attacks after the competition, not from the live leaderboard, so they measure how a frozen corpus of human attacks performs against each model rather than how quickly a motivated attacker solves a level.
End-to-end success is rare even when attacks are strong
The high ASR numbers do not mean most submissions worked. Fewer than 1% of all submissions managed to be retrieved, evade the defenses, and call the tool with the right arguments. Most attempts failed somewhere along that chain, which is exactly the complexity that template benchmarks skip when they present an attack directly to the model.
The paper reports a related pattern: detected attacks had substantially higher tool-call rates than undetected ones. In other words, the attacks that got past the defenses were not simply the ones that shouted loudest; the defense layer and the tool-call layer interact. Stacking defenses reduced success further, and in Phase 2 no submission solved the GPT-4o-mini sub-levels when all defenses were combined.
This is the paper's central tension. Human-written attacks are individually more dangerous than templates, yet a realistic pipeline with retrieval and multiple defenses still filters out the overwhelming majority of them.
Text classifiers were the weakest link
The challenge deployed five defenses: Spotlighting (delimiting and marking untrusted content), a production black-box classifier called Prompt Shield, an LLM judge prompted to flag injections, TaskTracker (a linear probe on activation differences before and after external data is processed), and a combination of all of them.
Text classifiers could not reliably separate legitimate email instructions from attacks, reaching only 60–68% recall. The authors explain this as a surface-feature problem: real work emails contain instructions, so a classifier matching known attack patterns will also flag benign mail or miss attacks that read like ordinary requests. The LLM judge did much better, reaching 99.4% recall, but the paper notes it required 5–6 times more attacker attempts than text classifiers — a cost trade-off rather than a free win.
The full stack reached 99.8% recall on attack submissions that called the send_email tool in Phase 1. That number describes detection of submissions that already reached the tool call; it is not the same as end-to-end ASR, and the two should not be substituted for each other.
Attacks transfer to coding and file tasks
To test whether the corpus is useful outside email, the authors ran two transfer experiments on GPT-4o. In the first, they kept the attack prompts but changed the agent scaffolding: a proof-of-concept coding agent with a different system prompt, context and tools, where send_email was available for alerts. Attacks reached 37% ASR there, against 58% in the original email setup.
In the second, an LLM paraphrased successful attacks to target new objectives. File upload reached 28% ASR and calendar deletion 45% ASR. The authors present these as proof of concept that the strategies, not just the wording, carry over — the semantic shift from summarizing email to analyzing a dataset did not eliminate success.
The transfer numbers are lower than the email baseline and come from a single target model, so they indicate direction and rough magnitude rather than a general cross-domain rate.
The taxonomy can seed new attacks, not just replay old ones
Replaying a frozen corpus tests whether past attacks still work; it does not test whether the underlying techniques can produce new ones. The authors distilled a taxonomy from successful attacks and used GPT-5.4 to generate 500 new prompts from it, without iterating against target-model feedback.
Ninety of those 500 succeeded against GPT-5.4-mini (18% ASR) and 10 against GPT-5.2 (2% ASR). Because there was no recursive refinement, the authors describe these as a measure of whether the extracted patterns are reusable, not as the maximum vulnerability of either model.
This distinction matters for anyone reading the headline ASR figures: the 18% and 2% are generation results under a fixed one-shot procedure, while the 32–58% figures are replay results on a large human corpus.
What the comparison cannot settle
The model comparisons are descriptive, not causal. Lower ASR is associated with instruction-hierarchy training and additional reasoning in these experiments, but architecture, training data, safety procedures and inference settings differ across proprietary models and cannot be separated. The paper says so directly.
Phase 2 also bundled several changes at once — an updated classifier, newer TaskTracker models, a revised judge prompt, defensive system instructions, input sanitization and a conformal blocklist — so the lower success rate cannot be attributed to any single component. The authors describe Phase 2 as exposing a stronger stack to another round of adaptive attacks, not as a controlled ablation.
The attack objective is also narrow: most sub-levels target sending an email, with Level 4 adding data exfiltration. That kept 64 sub-levels comparable and collected enough attacks per configuration, but it means the corpus is not a general benchmark of agent security across many tools.
Where the defensive priority shifts
The paper's most decision-relevant comparison is between defense types rather than between models. Production text classifiers were the weakest defense and the LLM judge was much stronger but far more expensive in attacker attempts, while stacking produced the best recall. For a deployment choosing where to spend, that ordering — stack first, then invest in contextual reasoning rather than pattern matching — follows from the measured recall and cost figures.
The authors' proposed direction is contextual detection: a defense that reasons about who an instruction is addressed to (the user or the assistant), which actions align with the current task, and whether the proposed action is what the user actually asked for. The evidence for this is indirect — it rests on the failure of surface classifiers and on attacks that read like ordinary work email — so it is a motivated design recommendation, not a tested result in this paper.
The practical reading is that a model's score on a template benchmark says little about its exposure to human-written email attacks, and that end-to-end pipelines with retrieval and stacked defenses remain the harder target even when individual attacks are strong.
Sources used in this answer
LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
A 208,095-prompt human-written red-teaming benchmark shows models scoring under 5% ASR on prior benchmarks still reach 32% (Meta SecAlign), 47% (GPT-5 mini) and 58% (GPT-4o) ASR, while fewer than 1% of all submissions succeeded end to end.