ReflectDrive-2: Mastery of Self-Editing in Discrete Diffusion Driving
ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving
ReflectDrive-2 is a Vision-Language-Action (VLA) autonomous driving planner that utilizes masked discrete diffusion to generate trajectories. By integrating a "decision-draft-reflect" pipeline with a self-editing mechanism called AutoEdit, it achieves a SOTA 91.0 PDMS on the NAVSIM benchmark with camera-only input.
Executive Summary
TL;DR: ReflectDrive-2 is a transformative VLA planner that reframes autonomous driving as a "decision–draft–reflect" process. By representing trajectories as discrete tokens and employing masked diffusion, it introduces AutoEdit—an in-place revision mechanism. Unlike previous works where "refining" is an afterthought, ReflectDrive-2 uses Reinforcement Learning (RL) to couple the drafting and editing phases, allowing the model to learn how to be corrected.
Positioning: This work moves beyond simple imitation learning (IL) by addressing the inherent error structures (longitudinal/lateral) of driving through a native, RL-aligned self-correction loop. It sets a new bar for camera-only performance on NAVSIM and efficiency on edge hardware (NVIDIA Thor).
The Problem: The Inherent Errors of "Copy-Paste" Planning
Imitation learning suffers from "covariate shift"—small errors in speed (longitudinal) or lane positioning (lateral) accumulate, eventually leading to catastrophic failures. Traditional planners are often "one-shot": once a trajectory is committed, there is no native way to fix it without re-running the entire model. Continuous diffusion planners attempt to fix this but often require 20+ denoising steps, making them too slow for real-time deployment.
Methodology: Decision, Draft, and Reflect
ReflectDrive-2 breaks down planning into three logical stages within a shared Transformer backbone:
- Goal Proposal (Decision): Instead of a single path, the model predicts a posterior of goal-point tokens, capturing diverse behaviors like "overtake" vs. "yield."
- Trajectory Drafting (Draft): Using masked discrete diffusion, it fills in the gaps between the ego-state and the goal in parallel.
- AutoEdit (Reflect): A token-to-token rewriting operator that corrects the draft.
The Secret Sauce: RL-Coupled Rollouts
The most significant insight is that supervised learning alone is insufficient for self-editing. Under supervised training, the editor learns to fix synthetic noise, but the drafter doesn't know how to produce "fixable" paths.
By applying RL (specifically group-relative policy gradients) to the full rollout (Draft → Edit), the terminal reward credits both stages. This forces the drafter to produce distributions where the post-edit score is higher than the pre-edit score.
Figure 1: The ReflectDrive-2 architecture utilizes a shared backbone for vision, language, and action tokens, enabling efficient KV reuse across the draft and reflect phases.
Experiments & Real-World Performance
ReflectDrive-2 was evaluated on NAVSIM, the high-fidelity nuPlan-based benchmark.
- The RL Boost: Without RL, AutoEdit only improved scores by 0.3. With RL, the gain jumped to 1.9 PDMS, proving that co-adaptation is the key to effective reflection.
- Closing the Gap with Human Performance: In the "Best-of-6" oracle setting, it achieves 94.8 PDMS (matching the human reference).
- Efficiency: Through "Alternating Step Decode" (ASD), the model reuses previous plans as "temporal drafts" for the current frame, slashing latency to 31.8ms.
Table 1: Competitive analysis on NAVSIM shows ReflectDrive-2 outperforming other VLA and end-to-end planners, particularly in Ego Progress (EP).
Visual evidence of "Reflection":
Figure 2: AutoEdit in action. Initial drafts (semi-transparent) that clip boundaries or misjudge gaps are pulled back into safe, high-reward regions (solid green) via in-place token rewriting.
Critical Insight & Future Outlook
Takeaway: ReflectDrive-2 proves that discrete diffusion is not just a trendy language modeling technique—it is a powerful tool for robotics. Its ability to perform "in-place" editing matches the way humans refine their own intentions.
Current Limitations: The use of discrete BEV coordinate tokens limits spatial precision to the "bin" size. Future iterations might integrate residual continuous offsets or higher-resolution vocabularies to handle tight urban maneuvers even more gracefully.
ReflectDrive-2 is a major step toward VLA models that don't just "predict" but "think and revise" in real-time.