CogDriver: Bridging the "Amnesia" Gap in Autonomous Driving with Cognitive Inertia

In the world of autonomous driving, the ability to see is no longer the bottleneck; the ability to remember and reason is. While Large Vision-Language Models (VLMs) have brought reasoning capabilities to the cockpit, they often suffer from what researchers call a "perpetual present." These models re-evaluate the entire world every few milliseconds, leading to dangerous indecision.

CogDriver shifts the paradigm from reactive prediction to temporally coherent planning by instilling "Cognitive Inertia"—the human-like persistence of intent that prevents a car from "changing its mind" every half-second during a complex maneuver.

TL;DR

CogDriver introduces a framework that gives AI drivers a "continuous thread of thought." By combining a new narrative-rich dataset (CogDriver-Data) with a memory-augmented agent (CogDriver-Agent), it achieves a staggering 63% relative improvement in Success Rate on the Bench2Drive closed-loop benchmark.

The Core Problem: The "Amnesiac" Driver

Current end-to-end systems are cognitively stateless. Imagine a car trying to overtake a truck. It sees the truck, decides to pass left, then a second later—forgetting its previous intent—sees a car far behind in the left lane and abruptly swerves back. This "decision jitter" is not just inefficient; it's unpredictable and dangerous.

The authors identify the root cause: a lack of Cognitive Inertia. Current models lack the internal "anchor" that links a sequence of observations into a single, evolving story.

Methodology: Building Internal Coherence

The authors solve this through two main pillars:

1. CogDriver-Agent: The Temporal Coherence Module (TCM)

Unlike standard Transformers that process tokens in isolation, the CogDriver-Agent features a Temporal Coherence Module. It uses:

• Geometric Propagation: Explicitly warping historical object queries into the current frame's coordinate system based on ego-motion.
• Motion-Conditioned Refinement: Dynamically adjusting features for fast-moving or occluded objects.
• State Reconciliation: A fusion layer that weighs "historical beliefs" against "new visual evidence."

Figure 1: The CogDriver-Agent architecture, showing how 3D perception and language commands are integrated into a stable world model.

2. CogDriver-Data: The "Continuous Why"

Existing datasets like DriveLM provide frame-by-frame rationales. CogDriver-Data provides narrative annotations. Using a Multi-View Spatiotemporal MLLM, the authors generated 227k narratives that explain the transition of scenes. Instead of saying "stopping for a red light," the model learns: "The light was green, now it is amber, so I am initiating a smooth deceleration to stop at the upcoming junction."

Experimental Results: Setting a New Standard

CogDriver doesn't just improve performance; it crushes previous baselines in interactive, closed-loop environments.

• Bench2Drive (Closed-Loop): Achieved a Driving Score of 78.21, outperforming the previous best (DriveAdapter) which stood at 64.22.
• Efficiency & Safety: On nuScenes, CogDriver achieved the lowest Average L2 error (0.34m) while maintaining a remarkably low infraction rate.
• VQA Reasoning: In visual question answering, CogDriver-Agent (a 7B class model) outperformed Qwen2-72B in understanding driving contexts, proving that specialized temporal training beats raw parameter scale.

Table 1: CogDriver-Agent vs. SOTA on primary closed-loop metrics.

Critical Insight: The Evolution of Rationale

The most fascinating part of this research is the Qualitative Analysis. The authors show that CogDriver’s internal reasoning "matures" over time. As a car approaches a turn, the rationale shifts from "recognizing a car ahead" to "anticipating the upcoming junction."

Figure 2: Visualization of the agent's evolving narrative across frames.

Conclusion: Toward Trustworthy Autonomy

CogDriver proves that for autonomous agents to interact safely in a human world, they must adopt human-like cognitive properties. By instilling cognitive inertia, CogDriver moves us closer to vehicles that don't just react to pixels, but understand the story of the road.

Future Outlook: The next step for this technology will likely involve scaling these narrative-based datasets even further and testing the limits of this "internal state" in extreme, long-tail edge cases (e.g., erratic pedestrian behavior or complex construction zones).