[NVIDIA 2026] AVO: Surpassing cuDNN and FlashAttention-4 via Autonomous Evolutionary Search

TL;DR

The era of human-exclusive GPU kernel optimization is coming to an end. NVIDIA researchers have unveiled Agentic Variation Operators (AVO), a framework where an AI agent—not a human engineer—iteratively designs, profiles, and debugs CUDA kernels. In a 7-day autonomous run on the new Blackwell (B200) architecture, AVO discovered attention kernels that are 10.5% faster than FlashAttention-4 and 3.5% faster than the highly secretive cuDNN.

Problem & Motivation: The "Black Box" of LLM-in-the-loop Search

Prior evolutionary search methods (like DeepMind's FunSearch) treated LLMs as simple mutation operators: the framework provides a "parent" code, and the LLM spits out a "child" code. This works for simple algorithms but fails for high-performance GPU kernels where performance depends on a delicate dance between register pressure, warp synchronization, and memory fences.

In these scenarios, an LLM needs to be more than a generator; it needs to be an engineer. It needs to read PTX documentation, understand the specific occupancy limits of the B200, and react to profiler results when a change regresses performance.

Methodology: The Agent as the Architect

AVO transforms the "Vary" operator of evolutionary search into a closed-loop autonomous agent.

Unlike previous pipelines, the AVO agent has:

1. Direct Agency: It decides when to read documentation, which part of the code to edit, and how to verify it.
2. Persistent Memory: It tracks the history of what worked and what didn't (the lineage).
3. Tool Use: It triggers the NVIDIA Profiler and compiler to get real-time feedback on its attempts.

If the agent gets stuck (stagnation), a separate Self-Supervision mechanism intervenes to suggest new optimization directions, ensuring the evolution doesn't stall for 7 days.

Hard Evidence: Surpassing Expert-Engineered Benchmarks

The results on the B200 GPU are staggering because the baselines—cuDNN and FlashAttention-4—represent the absolute pinnacle of human engineering.

• Multi-Head Attention (MHA): AVO achieved up to 1668 TFLOPS, setting a new SOTA.
• Grouped-Query Attention (GQA): Proving the "transferability" of the agent's logic, it took only 30 minutes for the agent to adapt its MHA optimizations to GQA, still beating the baselines by substantial margins.

Behind the Scenes: What did the AI actually optimize?

The paper reveals that the agent didn't just find "low-hanging fruit." It performed micro-architectural reasoning:

1. Branchless Rescaling: It replaced conditional branches with speculative, predicated math to avoid warp sync overhead (improving throughput by 8.1%).
2. Pipeline Overlap: It restructured the code so the "Correction Warp" could normalize data while the "MMA Warp" was still computing the next batch, overlapping previously serial tasks.
3. Register Rebalancing: It autonomously moved 8 registers from the "Softmax" group to the "Correction" group to eliminate spills to local memory.

Deep Insight & Conclusion: The End of "Human-in-the-Loop"?

AVO proves that when an LLM is given the right "agency" and tools, it can navigate search spaces that are too complex and tedious for human engineers. The 500+ trial-and-error attempts made by AVO in 7 days would have taken a human expert months of grueling labor.

Takeaway: The future of performance-critical software lies in "Agentic Search." As hardware becomes increasingly complex, the most efficient code will likely be written by agents that never sleep, never get bored, and have memorized every line of the hardware manual.