[CVPR 2026] Curious-VLA: Breaking the "Narrow Policy" Grasp in Autonomous Driving

TL;DR

Autonomous driving VLA (Vision-Language-Action) models often suffer from Narrow Policy (NP)—a state where they become "too confident" in a single path, killing exploration. Curious-VLA solves this by synthesizing diverse feasible trajectories during Imitation Learning and using a diversity-aware sampling strategy during Reinforcement Learning. The result? A new SOTA on Navsim (90.3 PDMS) and human-level performance in Best-of-N scenarios.

Background: The Curse of Over-Imitation

Most VLA systems follow a two-step recipe:

1. Imitation Learning (SFT): Learn to mimic human drivers.
2. Reinforcement Learning (RL): Fine-tune based on environmental feedback.

The authors identify a fatal flaw: Narrow Policy. Because standard SFT uses Cross-Entropy loss—which treats every path that isn't the Ground Truth (GT) as equally "wrong"—the model's probability distribution collapses. When you move to the RL stage (using algorithms like GRPO that rely on comparing a group of trajectories), the model samples 8 nearly identical paths. If there's no diversity, there's no "advantage" to calculate, and learning stops.

Methodology: Unleashing Exploration

Curious-VLA attacks the NP problem across both training stages.

1. Imitation Learning with FTE

Instead of forcing the model to only see one "perfect" human path, the authors introduce Feasible Trajectory Expansion (FTE).

• Data Expansion: They use a diffusion-based planner to generate multiple safe, physically valid paths for the same scenario.
• Step-wise Normalization: Waypoint scales vary wildly (0.5m at start vs. 50m at 4s). Curious-VLA normalizes each timestep independently, preventing far-horizon errors from drowning out near-horizon steering precision.

2. Diversity-Aware Reinforcement Learning

To keep the RL stage healthy, they introduce:

• ADAS (Adaptive Diversity-Aware Sampling): It filters out "boring" scenarios where the model's outputs are unimodal. It keeps only those where the model shows uncertainty, ensuring the policy gradient actually has signal to work with.
• SDR (Spanning Driving Reward): A focal-style reward function that stretches the value difference between a "good" and a "perfect" drive, making the model more sensitive to minor quality improvements.

Experimental Battlecard

The performance jump is most evident when checking the "Best-of-N" performance. While the average drive is SOTA, the model's ability to explore means that among 6 sampled paths, one is almost always a human-level "gold" trajectory.

Visual comparison showing Curious-VLA exploring multiple valid paths (blue) versus baseline collapse (red).

Critical Insight: Why it Works

The "Devil" is indeed in the Narrow Policy. By using Step-wise Normalization, the authors solved a hidden gradient imbalance problem where the model was essentially ignoring steering precision in favor of long-range goal reaching. By adding ADAS, they ensured the RL stage didn't waste "compute" on scenarios where the model had already reached a consensus, focusing instead on high-entropy "conflict" zones like intersections.

Conclusion & Future Work

Curious-VLA proves that the "exploit-explore" dilemma is the primary hurdle for end-to-end driving. While it hits SOTA, it currently operates in a "Slow Think" mode (1.57s latency). Moving forward, distilling this "exploratory wisdom" into faster, reactive "Fast Think" modules will be the next step toward actual vehicle deployment.

Takeaway: To make a model drive like a human, you must first let it imagine all the ways it shouldn't.