[CVPR 2026] SPATIALALIGN: Teaching Video Generators the "Physics" of Where Things Are

TL;DR

While modern Text-to-Video (T2V) models generate stunning pixels, they are often "spatially illiterate," failing to understand how objects should move relative to one another. SPATIALALIGN fixes this by introducing a geometry-based reward system (DSR-SCORE) and a stabilized DPO training strategy, boosting spatial correctness by over 450% without sacrificing aesthetic quality.

The Problem: The VLM Evaluation Mirage

Why do our best T2V models struggle when we ask a fox to "walk from the right to the left of a stump"? Most current benchmarks rely on Vision-Language Models (VLMs) like Qwen-VL or GPT-4o to evaluate performance. However, this paper reveals a startling "hallucination" in evaluation: VLMs tend to provide "Yes" answers regardless of the actual spatial reality in the video.

Figure 4: The weak correlation between VLM judgments and actual geometric truth shows that VLMs are currently unfit to act as "judges" for spatial movement.

Methodology: DSR-S CORE and Zeroth-Order DPO

To solve this, the authors moved away from "black-box" VLM judges and toward Euclidean Geometry.

1. DSR-SCORE: The Geometric Judge

By using GroundedSAM, the system tracks the bounding boxes of animals and objects frame-by-frame. It computes:

• SSR-Score: Measuring a static relationship (e.g., is the cat "on top"?) in a single frame.
• DSR-SCORE: Analyzing the "crossing" pattern of these scores over time to ensure a smooth transition from an initial state (left) to a final state (top).

2. Zeroth-Order Regularization

Standard Direct Preference Optimization (DPO) often suffers from likelihood displacement—where the model learns to "hack" the reward by making the "loser" sample's probability catastrophically low rather than making the "winner" sample better.

The authors introduce a Zeroth-Order (ZO) term: ${\mathcal{L}}_{ZO}=\Vert {ϵ}_{ref}-{ϵ}_{h}eta{\Vert }_2^2$ This acts as an anchor, ensuring the fine-tuned model doesn't drift too far from the high-quality distribution of the original reference model while it learns new spatial tricks.

Figure 2: The SPATIALALIGN pipeline: generation, geometric evaluation, and regularized DPO training.

Experimental Battleground

The model was put to the test against heavyweights like HunyuanVideo, CogVideoX, and Wan2.1.

SPATIALALIGN transformed Wan2.1 from a model that occasionally followed spatial prompts into the most spatially accurate model in its class, even beating the much larger HunyuanVideo in correctness.

Figure 5: Notice how SPATIALALIGN (right) maintains the animal's path correctly, while the baseline (middle) often loses track of the object-animal relationship.

Deep Insight: Beyond Prompt Overfitting

One of the most impressive findings is the Cross-Attention Map Analysis. The authors found that after fine-tuning, the attention scores between "spatial tokens" (like 'left' or 'top') and the "animal tokens" significantly increased. This suggests the model isn't just memorizing prompts; it's learning to bind spatial modifiers to the moving agents in the latent space.

Conclusion

SPATIALALIGN proves that for AI to understand our physical world, it needs more than just "aesthetic" training—it needs grounded, geometric feedback. By replacing fallible VLMs with rigorous spatial math, we can finally move closer to T2V models that can act as world simulators for robotics and physical reasoning.

Limitations: The system's performance currently relies on the tracking accuracy of GroundedSAM; complex scenes with heavy occlusion still pose a challenge for the metric.