[CoRL 2025] World2Act: Grounding VLA Policies in the Latent Dynamics of World Models

TL;DR

World2Act is a novel post-training framework that bridges the gap between World Models (WMs) and Vision-Language-Action (VLA) policies. Unlike prior works that trust the "hallucination-prone" pixels of generated videos, World2Act aligns the policy directly with the WM's internal latent dynamics via contrastive learning. Combined with a new skill-decomposition pipeline, it achieves SOTA performance on RoboCasa and LIBERO, while showing significant gains in real-world deployment.

The Core Challenge: Pixels Are Deceptive

Generative World Models are powerful, but they are not perfect. When a WM "imagines" a robot picking up a cup, it might synthesize a duplicate handle or cause the object to flicker (visual hallucinations).

• The Problem: Conventional post-training methods (like IDMs) attempt to map these noisy pixels back to actions. If the pixels are wrong, the action labels are garbage.
• The Complexity: Robotic tasks vary in length. A standard video model trained on 2-second clips fails when asked to perform a 30-second multi-stage kitchen task.

Methodology: Latent Alignment & Atomic Skills

The researchers at MBZUAI proposed a two-pronged solution:

1. Skill-Compositional World Model

Instead of generating one long, unstable video, they use an LLM (DeepSeek) to decompose high-level instructions into atomic skills (e.g., "reach," "grasp," "place"). They released RoboCasa-Skill and LIBERO-Skill datasets, which normalize the video length distribution, making the WM training far more stable.

2. World2Act Alignment

The framework operates in two stages:

• Stage 1 (Representational Bridge): They train Video and Action Adapters to map WM latents and robot actions into a shared space using a bidirectional InfoNCE loss. This ensures that "moving the arm left" in action-space and "visual movement to the left" in latent-space are mathematically synonymous.
• Stage 2 (Residual Post-Training): They freeze the heavy VLA (like GR00T-N1.6) and train a lightweight Residual Policy. This policy tweaks the base actions to better align with the WM's "visual plan" in the latent space.

Experimental Validation

World2Act was tested against a battery of SOTA models including π0, OpenVLA, and Cosmos Policy.

• Simulation Excellence: On RoboCasa, adding World2Act to GR00T-N1.6-ft boosted success rates to 72.6%, outperforming the best prior post-training baseline (VLA-RFT) by 1.6%.
• Robustness to Scaling: As shown in the scaling plots, World2Act improves monotonically with more synthetic data, whereas pixel-based methods (like DreamGen) show instability due to image-level noise.

Real-World Impact

The researchers deployed World2Act on a Franka Research 3 arm. Even when the World Model's visual output was slightly blurry (e.g., a missing drawer handle), the latent dynamics remained physically sound. This allowed the robot to achieve a 6.7% improvement in real-world success rates for tasks like picking cups and closing drawers.

Conclusion & Insights

The success of World2Act suggests that for Physical AI, the internal features of a world model are more valuable than its final rendered frames. By focusing on "what the model thinks will happen" (latents) rather than "what the model shows" (pixels), we can build far more resilient robotic agents.

Key Takeaways:

• Atomization is Key: Breaking tasks into skills is essential for long-horizon stability.
• Latents > Pixels: Latent alignment bypasses the "uncanny valley" of generated video artifacts.
• Efficient post-training: The residual policy approach allows for fast tuning (>250Hz inference) without destroying the base VLA weights.