OneDrive: Unifying the Mind and Action of Autonomous Driving in a Single Transformer Decoder

TL;DR

OneDrive is a breakthrough Vision-Language-Action (VLA) framework that eliminates the need for fragmented decoders in autonomous driving. By utilizing a single pretrained Transformer decoder to handle both autoregressive text reasoning and parallel structured prediction (perception & planning), it achieves State-of-the-Art (SOTA) safety and accuracy on nuScenes and NAVSIM while being significantly faster than current VLM-based competitors.

The Problem: The Great Architectural Divide

Most current autonomous driving systems are "architectural chimeras." They use one part of the brain for perception (typically a parallel DETR-like decoder) and another for reasoning (a sequential Large Language Model).

This fragmentation leads to two major issues:

1. Limited Backbone Reuse: You cannot fully utilize the power of a pretrained VLM if you keep bolting on new, randomly initialized decoders.
2. Information Bottlenecks: Forced separation between "seeing" (perception) and "thinking" (language) restricts the flow of information, making joint optimization unstable.

Research Insight: Attention Is All You Need (To Transfer)

The authors conducted a fascinating diagnostic study. They discovered that while Feed-Forward Networks (FFNs) in VLMs are highly specialized for text and transfer poorly to driving tasks, the Causal Attention mechanism is incredibly robust. It captures fundamental relationships between queries and visual features that are universal across tasks.

Table 1: Showing that reusing attention weights (Attn) is the key to cross-task performance.

Methodology: The Unified Sequence

OneDrive treats everything—images, 3D object queries, lane queries, and planning trajectories—as a single sequence of tokens.

1. Unified Token Representation

The system processes a sequence: [Visual Tokens, Detection Queries, Lane Queries, Planning Queries, Text Tokens]. Because perception and planning queries follow the visual tokens in a causal mask, they "naturally" attend to the visual context without needing explicit cross-attention layers.

2. Mixed Decoder Layers

To bridge the gap between parallel and sequential tasks:

• Backbone: Retains the pretrained VLM's causal attention.
• Adaptation: Adds a small self-attention block only among perception queries to allow them to "talk" to each other in parallel.
• Specialization: Replaces the text-heavy FFNs with lightweight, task-specific FFNs for detection and planning.

Figure 3: The OneDrive architecture showing the unified token flow through the mixed decoder layers.

Experiments: Safer and Faster

OneDrive was tested on nuScenes (open-loop) and NAVSIM (closed-loop), the gold standards for E2E driving.

• Safety First: OneDrive achieved a collision rate of 0.18% on nuScenes, significantly lower than previous action-based models like ColaVLA (0.23%) and SOLVE-E2E (0.30%).
• Efficiency: By using a truncated inference mode (using only the shallow layers for core driving tasks), OneDrive slashed latency by 40%, reaching 156ms for real-time deployment.

Table 2: OneDrive leading the pack in L2 error and Collision Rate.

Deep Insight: Why Does It Work?

The secret lies in the Multi-Stage Training. The authors start with perception-language pretraining, move to planning adaptation, and finish with joint fine-tuning. This "curriculum" allows the model to align its visual understanding with motor control without losing its "common sense" reasoning capabilities.

Interestingly, keeping the Text Loss enabled during driving training acts as a regularizer, preventing the attention modules from drifting too far from their pretrained semantic space.

Critical Analysis & Future Work

While OneDrive is a massive leap toward unified VLA models, it still faces challenges:

• Visual Bottleneck: Standard VLMs often downsample images, which can hurt fine-grained 3D detection.
• Scaling: The experiments used 1B and 2B models; how this architecture scales to 70B+ parameters remains to be seen.

Conclusion

OneDrive proves that we don't need complex, heterogeneous "franken-architectures" to achieve SOTA autonomous driving. A single, well-adapted VLM decoder can see, reason, and act—all within the same attention span.

OneDrive code and models are available at: https://github.com/Z1zyw/OneDrive