MotionCache: Harnessing Motion Dynamics for Ultra-Fast Autoregressive Video Generation

TL;DR

Autoregressive video generation is the new frontier for long-duration video synthesis, but its iterative nature makes it prohibitively slow. MotionCache breaks the bottleneck by shifting from "whole-frame" caching to "token-wise" caching. By using a lightweight motion proxy, it identifies which parts of a frame (like a moving athlete) need recomputing and which parts (the background) can be safely reused. The result? Up to 7.26× speedup with virtually no loss in quality.

Background: The Price of Infinite Frames

The shift towards Causal Diffusion-Forcing (CDF) and autoregressive designs has solved the memory explosion of long videos, but the inference time remains the "elephant in the room." Generating a mere 7-second video can take nearly 30 minutes on a high-end A800 GPU. Current acceleration methods like FlowCache or TeaCache act like a sledgehammer—they either compute the whole frame/chunk or skip it entirely. This "binary" approach fails because video content is spatially heterogeneous: backgrounds stay still while subjects move.

Motivation: The Mathematics of Motion

The authors derive the Residual Inconsistency Principle, proving that caching error is strictly proportional to the difference between the current residual and the cached one.

The breakthrough Insight: Intra-chunk frame differences are a high-fidelity surrogate for this residual instability. In simple terms, if a pixel changes a lot between frame $f$ and frame $f+1$, the model's prediction for that pixel is likely to be unstable and requires a fresh calculation.

Methodology: The Coarse-to-Fine Strategy

MotionCache doesn't just start skipping tokens immediately. It follows a logical two-phase approach:

1. Phase 1: Structural Warm-up: The first few blocks of a video are critical for semantic layout. The model performs full chunk-level updates to ensure the "skeleton" of the video is solid.
2. Phase 2: Motion-Aware Refinement: Once the structure is stable, the model calculates a "Motion-Aware Token Importance" map.

Tokens with high motion accumulate error budget faster and trigger a recomputation, while static tokens are simply retrieved from the Residual Cache.

Figure: MotionCache replaces the "all-or-nothing" approach with a fine-grained selection mask.

Experiments and Results

The framework was tested on two heavyweights: SkyReels-V2 and MAGI-1.

• SkyReels-V2: MotionCache-fast hits a 7.26× speedup. Unlike FlowCache, which causes "melting" faces or extra fingers in high-speed settings (e.g., a person tasting beer), MotionCache maintains anatomical correctness.
• MAGI-1: It achieves 2.07× acceleration. In complex scenes like an elephant walking, MotionCache is the only method that preserves fine details like the elephant's tusks, which disappear in other accelerated versions.

Table: Quantitative comparison showing superior PSNR and SSIM for MotionCache at higher speedup ratios.

Critical Insight: Why it Works

The success of MotionCache lies in its Soft-mapping floor ($\alpha$). By ensuring that even static tokens get a "background" update rate, it prevents the background from becoming a blurry mess over long sequences—a common failure mode in previous research.

Conclusion

MotionCache marks a transition from "temporal redundancy" to "spatial-temporal redundancy." It proves that the most efficient way to generate video isn't to work harder on every pixel, but to work smarter by focusing on where the action is. This is a vital step toward bringing high-quality, long-form video generation to real-time applications.

Limitations: The method relies on a warm-up phase; if the initial structural denoising is flawed, the subsequent motion-tracking cannot fix it. Future work could potentially use lightweight motion vectors from video encoders to further reduce the cost of importance estimation.