TL;DR

Training robots to handle the messiness of the real world requires massive, diverse datasets that are nearly impossible to collect physically. WorldComposer solves this by turning a single 360° panorama into a "multiverse" of high-fidelity simulations. By generating Digital Cousins—variations of real-world scenes with different layouts and textures—it provides a scalable data engine that boosts robot generalization and offers a simulation environment so accurate that its results correlate 91% with real-world trials.

The Motivation: Why "Twins" are Not Enough

In the quest for generalizable robot policies, researchers have long used "Digital Twins"—exact virtual replicas of a specific real-world setup. While useful for debugging, Digital Twins suffer from overfitting. If a robot only learns to pick up a cup in one specific kitchen, it fails the moment the wallpaper changes or the microwave is moved two inches.

The authors identify that the real bottleneck isn't just "sim-to-real," but the lack of environmental diversity within those simulations. We don't just need mirrors of the world; we need "cousins" of the world that explore the "what if" of scene configurations.

Methodology: The Generative Real-to-Sim Pipeline

WorldComposer operates through a three-stage workflow that bridges the gap between a simple photo and a complex, navigable house.

1. From Panorama to Digital Cousins

Using the Marble world model, the system takes a single panorama and reconstructs:

• Visual Layer: 3D Gaussian Splatting (3DGS) for photorealistic rendering.
• Physical Layer: A collision mesh for interaction.

The "magic" happens with Prompt-Driven Editing. By giving a command like "a kitchen with wooden textures," the system generates a "Digital Cousin" that maintains the structural logic of the room but changes the visual and semantic distribution.

Figure 1: The framework transitions from a real-world capture to a precise Twin, and then to diverse Cousins via LLM-guided prompts.

2. Multi-Room Stitching

Since one panorama only sees one room, WorldComposer introduces a pipeline to stitch multiple rooms together. It uses SuperPoint and LightGlue for coarse alignment and Iterative Closest Point (ICP) for geometric refinement, ensuring a seamless, navigable floorplan for long-horizon tasks like navigation.

3. Populating the World with Physics

Static rooms are useless for manipulation. WorldComposer populates these scenes with a library of:

• Rigid Objects: Cups, plates (stable grasping).
• Articulated Objects: Microwaves, drawers (kinematic chains).
• Deformable/Fluids: Cloth, water (Position-Based Dynamics & FEM).

Experiments & Results: Proving the Fidelity

The researchers put WorldComposer to the test across 7 complex tasks, including folding cloth and pouring water.

The Scaling Law of Cousins

The most impressive result is the scaling effect. By incrementally adding up to 1,000 Digital Cousin trajectories to just 50 real-world samples, the success rate on the most difficult "Unseen Scene & Object" task skyrocketed from 10% to 85%.

Figure 2: Performance gains as the volume of generated "Digital Cousin" data increases.

Sim-Real Correlation

To prove this isn't just "playing in a sandbox," they mapped the performance of four different policy architectures (ACT, Diffusion Policy, SmolVLA, and π0) in both sim and real. The result was a Pearson correlation of 0.91, meaning if a policy improves in WorldComposer, it is almost certain to improve in the real world.

Figure 3: The tight alignment between simulation success and real-world results across multiple tasks.

Critical Insights & Conclusion

WorldComposer marks a shift from manual simulation design to AI-generated simulation.

• Takeaway: Diversity is a first-class citizen. The "Digital Cousin" concept effectively automates Domain Randomization, but in a way that is semantically grounded and physically consistent.
• Limitations: Currently, the system relies on LLMs for common-sense object placement and Marble for scene global meshes. Future work targets instance-level decomposition and solving texture "seams" at room junctions.

This framework essentially creates a "Data Engine" for robots—where a single afternoon of panoramic photography can provide enough training data to prepare a robot for thousands of unique, unseen homes.