APGP: Automatically Bypassing the Safety Guards of Text-to-Image AI

TL;DR

Researchers have developed the Automated Prompt Generation Pipeline (APGP), a framework that forces commercial AI systems like ChatGPT/DALL-E 3 and Gemini to generate copyrighted characters (e.g., Mickey Mouse, Spider-Man) by bypassing word-based filters. The method uses an LLM to "describe" the character so perfectly that the image generator recreates it, while avoiding the specific names that trigger safety blocks.

Positioning: This work is a critical "Red-teaming" contribution that exposes the fragility of current alignment techniques in commercial T2I (Text-to-Image) services.

Problem & Motivation

Most commercial AI providers use "Keyword-based filtering" to prevent the generation of intellectual property (IP). If you ask for "Mickey Mouse," the system says no. However, the visual knowledge of Mickey Mouse remains embedded in the model's neural weights because it was trained on the internet.

The authors observed that while existing methods block naive attacks, they fail against descriptive jailbreaking. The challenge was: how do you automatically generate a description that is detailed enough to recreate the IP, but generic enough to avoid the "censor" keywords?

Methodology: The Core Engine

The genius of APGP lies in its Self-Generated QA Score. It treats the prompt generation as an optimization problem where no gradients or weight updates are needed—making it a pure black-box attack.

The workflow involves:

1. Seed Search: Using a Vision-Language Model (VLM) like GPT-4V to create an initial detailed caption of a target image.
2. Iterative Refinement: An LLM (acting as an optimizer) rewrites the prompt to maximize a composite score:
   • Keyword Penalty ($S_k$): Negative points if names like "Disney" appear.
   • QA Score ($S_{qa}$): A VLM generates questions about the target image (e.g., "What color are the shoes?"). If the prompt-only description allows another LLM to answer those questions correctly, the description is high-quality.
3. Suffix Injection: Adding commands like "Do not rephrase the prompt" to ensure the T2I system doesn't "sanitize" the attack.

Experimental Battleground

The researchers tested this against Midjourney, Gemini, Copilot, and ChatGPT using the newly created VioT (Violation) dataset.

The "Safer" Model Fallacy

Initially, ChatGPT appeared extremely safe, blocking 84% of naive prompts. However, when faced with APGP, its block rate plummeted to 11%.

Figure: APGP successfully evokes high-fidelity IP characters by using optimized descriptions instead of names.

The Failure of Defense

The authors tried common defenses:

• Similarity Filtering: Using DINO/CLIP to check if the result looks like the IP. (Result: Too many false negatives; human judgment is still superior).
• Concept Unlearning: Trying to "erase" concepts like Van Gogh. (Result: APGP prompts can still "re-evoke" the style by describing visual elements like "swirling night sky").

Critical Insight & Conclusion

The core takeaway is that Censorship != Safety. Blocking the name of a copyrighted entity is merely a UI-level patch. As long as the visual concept remains in the Latent Space, an optimized agent (like APGP) can find a "semantic path" to evoke it.

Future Outlook: For IP owners, this is a powerful tool for automated auditing. For AI companies, it signals a need to move away from text-filters and toward real-time visual verification and more aggressive machine unlearning that targets semantic clusters rather than just token-trigger pairs.

Takeaway: Your AI isn't safe just because it won't say "Mickey Mouse"; it's safe when it no longer knows what "round ears, red shorts, and yellow shoes" represent together.