[CVPR 2025] Training-free Motion Factorization: Solving the Homogenization Trap in Video Generation

TL;DR

Compositional Video Generation (CVG) often suffers from "boring" or "uniform" motion where everything in the scene moves (or stays still) in exactly the same way. This paper introduces a Training-free Motion Factorization framework. By decomposing motion into Static, Rigid, and Non-rigid categories and using an LLM-driven Motion Graph, the authors enable existing video models to generate complex, multi-instance scenes with distinct, physically plausible dynamics—all without a single step of extra training.

The "Motion Homogenization" Crisis

While heavyweights like Sora or CogVideoX generate visually stunning frames, they often fail the "compositional" test. If you ask for "a static mountain while a car drives by and a person dances," current models often produce a scene where either everything jitters slightly or the person moves as stiffly as the car.

The authors identify two root causes:

1. Linguistic Ambiguity: LLMs often struggle to translate "dancing" into precise pixel-level coordinate changes over time.
2. Uniform Guidance: The internal attention mechanisms of Diffusion models treat all pixels equally, failing to enforce different physical "laws" for different objects.

Methodology: Planning Before Generation

The framework operates in two distinct phases: reasoning the "What/Where" and guiding the "How."

1. Structured Motion Reasoning (SMR)

Instead of asking an LLM to "give me boxes for a person dancing," the system builds a Motion Graph.

• Nodes: Represent instances (Person, Car) with motion labels.
• Edges: Represent interactions (Person inside Car).

This graph acts as a world-model-lite, allowing the LLM to reason about velocities and accelerations to produce a coherent Spatial-Temporal Layout.

2. Disentangled Motion Guidance (DMG)

This is where the physical intuition comes in. Once the layout is set, the framework injects three types of guidance into the diffusion process:

• Reference Conditioned Guidance (Static): Anchors features to a stable reference frame to eliminate the "background flicker" common in AI videos.
• Geometric Invariance Guidance (Rigid): Uses a shape template to ensure that a moving car doesn't morph its geometry as it travels across the screen.
• Spatial Deformation Guidance (Non-rigid): For complex subjects like dancers, it uses pixel-wise displacement fields to ensure local parts move fluidly while following the overall trajectory.

Experiments & Results: Quantitative Superiority

The authors evaluated their method on two new benchmarks, CVGBench-m and CVGBench-p, derived from real-world datasets like Panda-70M.

The results show a "clean sweep" across quality metrics:

• Subject Consistency: Improved by ~7% over other regional prompting methods.
• Dynamic Degree: A massive leap in VideoCrafter-v2.0 (from 38.40% to 82.21%), proving the model is now capable of much larger, more intentional motions.

Depth Insight: Why It Works

The "Secret Sauce" here is the pixel-wise consensus. By generating a shape template and enforcing cross-frame interaction only within geometrically aligned regions, the authors overcome the "shape-shifting" nature of current DiT and U-Net architectures. Furthermore, by using Llama-3-70B to "reason" the motion graph, they capitalize on the emergent physical reasoning capabilities of LLMs that smaller models (8B) lack.

Critical Analysis & Conclusion

Takeaway: This paper is a significant win for modular AI. It proves we don't need to retrain massive video models to make them smarter; we just need better "directors" (LLMs) and "constraints" (Motion Guidance).

Limitations:

• The model still struggles with rare semantic concepts (e.g., "Dendroid") and complex facial emotions, likely because these are absent from the underlying diffusion model's latent space.
• Future work could integrate camera motion (pans/tilts) to separate global ego-motion from instance identity.

Final Thought: As we move toward interactive AI agents, the ability to factorize motion is the first step toward true "physics-aware" video generation.