[CVPR 2025] SenCache: Breaking the Heuristic Barrier in Diffusion Acceleration with Sensitivity-Aware Caching

TL;DR

Diffusion models are notoriously slow, requiring hundreds of sequential network passes. While "caching" (reusing previous outputs) is a popular fix, most current methods rely on "gut-feeling" heuristics that break easily. SenCache changes the game by introducing a theoretically grounded caching rule based on network sensitivity. It predicts how much the model output would change before actually running it, allowing for aggressive yet safe speedups (up to 41% latency reduction) without retraining.

The "Heuristic" Headache: Why Prior Caching Fails

To speed up Video Diffusion Transformers (DiTs), researchers have tried to skip steps by looking at "proxies":

1. TeaCache looks at changes in the timestep embedding.
2. MagCache looks at the magnitude of residuals.

The problem? These are partial truths. The authors of SenCache prove that the denoiser output is a function of both the noisy latent ($x_t$) and the timestep ($t$). If you only track one, you miss the "drift" in the other, leading to those nasty visual artifacts we see in aggressive acceleration. Prior methods are static; they don't care if your video is a still sunset or a chaotic car chase.

Methodology: The Physics of Caching

The core insight of SenCache is treating the neural network as a locally smooth function. Using a first-order Taylor expansion, the authors quantify the potential error of skipping a step:

$f_{heta}(x_{t+\Delta t},t+\Delta t)-f_{heta}(x_t,t)\approx J_x\Delta x_t+J_t\Delta t$

Where:

• $J_x$: Sensitivity to the noisy image/video.
• $J_t$: Sensitivity to the "time" progress.

By pre-computing these sensitivities on a tiny calibration set (just 8 videos!), SenCache creates a Sensitivity Score ($S_t$). During inference, if $S_t$ is below a threshold $\epsilon$, the model simply reuses the last output. It’s an "early exit" strategy grounded in calculus.

Figure 1: SenCache adaptively decides to refresh or reuse the cache based on the calculated sensitivity score.

Visual Evidence: Efficiency vs. Quality

The researchers tested SenCache on the "Big Three" of open-source video generation: Wan 2.1, CogVideoX, and LTX-Video.

Key Findings:

• Superior Robustness: At 21 NFEs (Number of Function Evaluations), SenCache maintains a PSNR of 29.14, while previous SOTA methods drop to 28.36 or lower.
• Sample Adaptivity: Unlike static methods, SenCache allocates more compute to complex frames and skips more during "smooth" parts of the generation.
• Calibration Efficiency: You don't need a supercomputer to calibrate it. 8 diverse videos provide the same accuracy as 4,096 videos for estimating sensitivity profiles.

Table 1: Quantitative comparison showing SenCache outperforming TeaCache and MagCache across diverse metrics.

Critical Analysis: Is it the Universal Solution?

SenCache is a massive step toward principled acceleration. It moves the conversation from "let's try this trick" to "let's measure the manifold's curvature."

The Caveats:

• The "n" Limit: The authors found that caching for more than 2-3 steps consecutively leads to "drift" because the first-order approximation becomes stale. You still need to "refresh" the model periodically.
• Model-Specific Sensitivity: As seen in Figure 4 of the paper, different models have radically different sensitivity profiles. Wan 2.1 is sensitive at the end of the process, while LTX-Video is sensitive at the beginning. This proves that "one-size-fits-all" schedules are fundamentally flawed.

Figure 2: Jacobian analysis showing how sensitivity $(J_x,J_t)$ evolves across timesteps for different models.

Conclusion: The Path Forward

SenCache provides a "plug-and-play" speed boost for any diffusion model without the need for expensive distillation or retraining. Its value lies in its simplicity and theoretical rigor. For developers deploying LMMs (Large Multimodal Models) in production, this sensitivity-aware approach offers a predictable way to trade off a tiny bit of math for a lot of speed.

Takeaway: In the race to real-time generative video, the smartest way to run faster isn't to move your legs quicker—it's knowing when you can afford to close your eyes and keep gliding.