[CVPR 2026] WorldCache: Accelerating World Models for Free via Heterogeneous Token Caching

1. TL;DR

Diffusion-based world models are powerful but computationally expensive due to iterative denoising. WorldCache is a training-free framework that delivers up to 3.7× end-to-end acceleration while maintaining 98% quality. It works by identifying which "tokens" (parts of the scene or modalities) are easy to predict and which are "chaotic," applying adaptive computation only where it is physically necessary.

2. Background: The Problem with One-Size-Fits-All Caching

"World Models" (like Sora or Voyager) don't just generate images; they simulate physical dynamics across multiple modalities (e.g., RGB video and Depth). Previous acceleration methods like TeaCache or DeepCache treat every pixel/token equally, assuming they all evolve at the same speed.

However, the authors found two terminal flaws in this approach for world simulation:

• Token Heterogeneity: Multi-modal tokens (RGB vs. Depth) have wildly different trajectories.
• Temporal Non-stationarity: A world simulation might be smooth for 40 steps and suddenly become volatile during a complex motion segment. In these moments, a small minority of "hard" tokens dominates the error, causing the entire simulation to drift or "blur" if uniform skipping is used.

3. Methodology: CHTP and CAS

To solve these, WorldCache introduces two core mechanisms inspired by physical kinematics.

A. Curvature-guided Heterogeneous Token Prediction (CHTP)

Instead of guessing, the model calculates the Curvature Score ($\kappa$) of each token's feature trajectory: ${\kappa }_i=\frac{\Vert {\mathbf{a}}_{t_0,i}{\Vert }_2}{\Vert {\mathbf{v}}_{t_0,i}{\Vert }_2^2+\epsilon }$ Where $\mathbf{v}$ is velocity and $\mathbf{a}$ is acceleration within the feature space.

• Stable Tokens: Low $\kappa$. These are simply reused.
• Linear Tokens: Moderate $\kappa$. These use 1st-order linear extrapolation.
• Chaotic Tokens: High $\kappa$. These use a Hermite-guided damped predictor to prevent the model from "shooting off" into noise during sharp direction changes.

B. Chaotic-prioritized Adaptive Skipping (CAS)

Standard adaptive skipping uses raw feature differences, which fail because feature scales change across timesteps. WorldCache uses a dimensionless drift indicator. By multiplying curvature with displacement, they create a scale-invariant score. The system monitors only the chaotic tokens; when their accumulated drift hits a threshold, a "Full" backbone evaluation is triggered.

4. Experimental Results: Speed without Sacrifice

Tested on HunyuanVoyager-13B and Aether-5B, WorldCache proved to be the most efficient "training-free" method to date.

Regarding 3D Reconstruction, WorldCache maintained nearly perfect Pose and Depth accuracy (ATE 0.184 vs 0.177 Baseline) while other caching methods like TaylorSeer saw significant degradation in geometric consistency.

Visual results show that WorldCache prevents the "color noise" and "boundary bleeding" common in other caching methods.

5. Critical Analysis & Takeaway

The genius of WorldCache lies in its Curvature Score. It acknowledges that in a multi-modal world, the "bottleneck" is not the average performance, but the most chaotic 10% of tokens.

Limitations: While training-free, the method handles 3 historical FULL outputs in its buffer, which adds a tiny amount of memory overhead, though negligible compared to layer-wise methods.

Conclusion: This work provides a scalable blueprint for interactive world simulators. By prioritising "chaotic" tokens, we can simulate complex worlds at interactive speeds without the "blurry" artifacts that plagued earlier diffusion accelerators.