[arXiv 2026] WanderDream: Scaling World Models for Emulative Simulation and "What-If" Reasoning

TL;DR

WanderDream is a pioneering framework that allows AI agents to "mentally explore" environments. By generating consistent panoramic video trajectories from a single observation to a target situation, it enables models to answer complex spatial questions without moving a single inch. It bridges the gap between seeing and understanding via high-fidelity visual imagination.

Problem & Motivation: The "Inaccessibility" Barrier

In the real world, robots and humans often cannot afford to "explore first and ask questions later." A robot may be blocked by physical obstacles (stairs, uneven terrain), and a visually impaired person might feel unsafe moving into an unperceived area.

Current Multimodal Large Language Models (MLLMs) are good at describing what is already in the frame, but they struggle with situated reasoning—the ability to project oneself into a future position and reason from that viewpoint. Previous benchmarks like SQA3D focused on static scenes. WanderDream introduces the concept of Emulative Simulation: the cognitive ability to simulate an experience to answer "what-if" questions.

Methodology: The Imagination Engine

The authors propose a two-pronged dataset:

1. WanderDream-Gen: 15.8K panoramic videos. It simulates robotic navigation (landmark-based) and human movement (Dijkstra's shortest path on 3D Probabilistic Roadmaps).
2. WanderDream-QA: 158K question-answer pairs generated via GPT-5 and Set-of-Mark (SoM) prompting. It covers three phases: Start State, Path, and End State.

Architecture: Sequential & Closed-Loop

The paper evaluates different "World Models" (Wan, CogVideoX, HunyuanVideo) as the imagination core. They use two main frameworks:

• Sequential: A World Model generates the whole video first, then an MLLM (like Qwen3-VL) reasons over it.
• Closed-Loop: Step-by-step action-prediction and imagination (similar to MindJourney).

Fig 1: The agent imagines a trajectory (blue) from the current state (green) to answer questions about the future path.

Experiments and Results

The study provides a definitive answer to a crucial question: Is imagination necessary? The data shows that for "End State" reasoning, having intermediate imagined frames ($s_{\Delta 5}$) is superior to just having the start and end images. This proves that the process of imagining the path helps the model ground the spatial relationships of the destination.

Key Metrics:

• World Model Performance: Wan2.1 (fine-tuned with LoRA) achieved the best temporal coherence (FVD).
• Reasoning Accuracy: Fine-tuning CogVideoX1.5 on WanderDream data led to the highest end-state reasoning scores on ScanNet++ (47.5).
• Sim-to-Real: Even when trained on "perfect" simulated trajectories, the models transferred surprisingly well to real-world, jittery head-mounted camera footage.

Fig 2: Frameworks for simulation: (a) Prompt extension, (b) Fine-tuning (WanderDream strategy), and (c) Closed-loop.

Critical Analysis & Conclusion

While WanderDream is a massive step forward, it highlights a remaining bottleneck: Latency. Generating these "mental videos" currently takes anywhere from 35 to 283 seconds per trajectory. For real-time human assistance, this must be optimized.

Takeaway: The future of spatial AI isn't just better perception; it's better prediction. By treating World Models as a "mental sandbox," WanderDream paves the way for agents that can understand the consequences of their actions before they take them.

Future Directions

The authors suggest moving toward Unified Video-and-Text Modeling, where a single end-to-end model can imagine the video and "speak" the answer simultaneously, further reducing the architectural complexity of these sequential pipelines.