[arXiv 2026] LiveWorld: Breaking the Static-World Assumption in Generative Video Models

TL;DR

Current video world models have a "blind spot": they freeze the world the moment you look away. LiveWorld introduces a paradigm shift by decoupling World Evolution from Observation Rendering. By deploying virtual "Monitors" to simulate unobserved entities in the background, it ensures that when you turn your camera back, the world has naturally progressed—bridging the gap between 2D snapshots and true 4D persistent simulation.

Problem: The "Frozen" Reality of AI World Models

Most modern world models (like Genie or Sora-based architectures) treat the world as a series of 2D snapshots. While they can handle camera movement, they implicitly assume that if an object isn't on screen, it doesn't change.

In technical terms, they conflate the evolution function $\mathcal{E}$ and the rendering function $\mathcal{R}$ into a single black box. This leads to the out-of-sight dynamics problem:

• Physical Inconsistency: A person walking away and then coming back appears exactly where they were left, regardless of elapsed time.
• State Stagnation: Re-visiting a scene retrieves "frozen" memory rather than an updated reality.

Methodology: Decoupling Evolution and Rendering

LiveWorld tackles this by approximating the world state ${\mathcal{W}}_t$ into two distinct components:

1. Static Background ($M_{static}$): Accumulated into a 3D point cloud via SLAM-like fusion.
2. Dynamic Entities ($M_{dyn}$): Sparse active objects that retain their temporal dimension.

The Monitor Mechanism

When the system detects a dynamic entity (like a dog or a car), it registers a virtual Monitor at that location. Even when the observer leaves, the Monitor continues to "fast-forward" the entity's actions in the latent space.

Unified Video Backbone

The authors use a unified State-Conditioned Video Diffusion model. It serves two roles:

• As an Evolution Engine ($G^{evo}$): Simulating local updates for the monitors.
• As a Renderer ($G^{render}$): Projecting both the static and updated dynamic point clouds into the observer's current viewpoint for final frame synthesis.

Experiments: Measuring the Unseen

To evaluate this, the authors introduced LiveBench, focusing on "revisiting" scenarios.

Performance Gains

Compared to SOTA models like Spatia or GameCraft, LiveWorld maintains significantly better foreground identity and event logic. In a "Same-Pose Revisit" after a long temporal gap, LiveWorld's dynamic entity consistency (CD${}_{fg}$) is nearly 40x better than baselines that lack explicit evolution.

Ablation Insights

• Without Event Evolution: The model reverts to a standard camera-control model where objects freeze.
• Without Spatial Memory: The scene suffers from "ghosting" and jitter during camera movement because the 2D frames aren't grounded in 3D space.

Critical Analysis & Conclusion

LiveWorld represents a critical step toward "Persistent Reality" in AI. Its success stems from acknowledging that memory should be 4D (Spatio-temporal) rather than 3D (Spatial-only).

Limitations:

• The number of active monitors is currently limited (M=3), which might be insufficient for highly crowded city-scale simulations.
• It relies on entity detection; if the detector misses an object, it remains frozen.

Future Outlook: This architecture is a blueprint for the next generation of simulators for autonomous driving and embodied AI, where the agent must reason about where a hidden pedestrian might be now, not where they were five seconds ago.