Why world-action models were hard to compare
World-action models combine prediction of future observations with robot control. The paper argues that existing systems change several things at once: the video backbone, how video and action streams interact, what supervision is used, and how inference is ordered. When all of these move together, a reported success rate cannot tell a reader which design choice produced the gain.
OpenWAM's response is to hold the visual foundation fixed and make the interaction pattern the variable. Every downstream variant starts from the same causal robot-video checkpoint, shares the tokenization and objective family, and is evaluated with the same protocol. That turns a comparison of whole systems into a comparison of interaction programs, which is the framework's main contribution as a testbed.
One backbone, four video-action interaction programs
The backbone begins as Wan2.2-5B, a general video model, and is further pretrained on more than 10,000 hours of robot and human-interaction video without action labels. Because the original model is not temporally causal, the authors add chunk-causal prediction: a future chunk may attend to the current observation, earlier observations, and previously generated chunks, but never to later chunks, while tokens inside a chunk are denoised jointly. This gives the model a fixed temporal direction, which sequential prediction needs.
An action branch is then attached through a Mixture-of-Transformers architecture. Video and action trajectories are encoded as temporally aligned streams, each processed by its own transformer branch with its own normalization, projection, and feed-forward parameters, and the two meet in a joint attention over packed video and action tokens. The paper describes four interaction programs on this shared structure: joint generation, video-then-action (VTA), action-then-video (ATV), and decoupled generation. Temporal causality and cross-modal interaction are treated as separate dimensions: causality decides which times are visible, while the interaction program decides which streams communicate.
Pretraining gains outweigh the architecture change
The clearest evidence for the shared foundation comes from an initialization ablation on LIBERO-Long closed-loop success. With downstream architecture and training held fixed, random initialization reaches 20.0% for VTA and 27.2% for joint generation; the original Wan2.2 checkpoint reaches 68.4% and 62.2%; the robot-video pretrained checkpoint reaches 97.8% and 96.6%. The jump from the original video model to the robot-video model is therefore far larger than anything the interaction design contributes in the same table.
A second ablation isolates the Mixture-of-Transformers choice. All variants here start from the robot-video checkpoint. A non-MoT version, which processes video and action tokens in the same pretrained DiT, already reaches 92.8% for VTA and 93.6% for joint generation. MoT raises these to 97.8% and 96.6%, gains of 5.0 and 3.0 percentage points. The paper's own reading is that the shared-DiT model is already strong and MoT adds a further, smaller improvement. For a reader deciding where to spend effort, the ordering is explicit: robot-video causal pretraining first, modality-specific experts second.
Frozen dynamics components survive a new task
The framework also supports component-level composition: an inverse dynamics model (IDM) turns a supplied visual transition into actions, and a forward dynamics model (FDM) predicts the visual consequence of supplied actions. These local-context models condition only on the boundary observation, proprioception, and either the future video or the action sequence; they do not use task language or pre-start history.
The transfer experiment asks whether such a component can be reused after only the video predictor is adapted. Four held-out LIBERO-90 tasks are used, the IDMs stay frozen, and the video predictor is tuned per task. A local-context IDM trained on counterfactual data and demonstrations reaches 84.0% mean success, against 47.0% for a full-context IDM with mixed supervision and 21.5% for a local-context IDM trained only on demonstrations. The per-task spread is the important detail: the mixed local-context model scores 74%, 86%, 88%, and 88% across the four targets, while the demo-only local-context model scores 36%, 50%, 0%, and 0%. The task-tuned VTA policy itself reaches 93.0% mean, so the frozen component narrows but does not close the gap to a policy trained for the target task.
The same components behave differently on the source task. On LIBERO-Long with the LIBERO-Long video predictor, a full-context IDM in the native VTA setup reaches 97.8%, a local-context IDM trained only on demonstrations reaches 25.8%, and local-context IDMs trained on counterfactual-only or mixed data reach 90.6% and 94.4%. Local context is therefore not a free substitute for full context; it becomes useful when counterfactual transitions supply the missing supervision.
Counterfactual transitions supply what demonstrations lack
Demonstrations give roughly one successful continuation per visited state, which is enough for imitation but thin supervision for dynamics models that must relate alternative actions to alternative outcomes. The authors build LIBERO-Long-CF by branching from task-relevant states in the ten LIBERO-Long tasks and executing modified future action sequences. It contains 32,000 counterfactual segments, 128 future controls each, and 4.10 million control records against 138,090 controls in the 500 source demonstrations, about 29.7 times more local transition supervision.
The interventions are concrete: stopped or rescaled motion, reversed or redirected translation, axis biases and pulses, randomized arm controls, and gripper changes. Among characterized segments, 90.4% contain gripper contact with an object or fixture, 50.0% contain a detected grasp, and 72.3% alter object configuration relative to the same-start reference continuation. These statistics describe the collected data, not a downstream success rate, and they are what makes the dataset a test of local dynamics rather than a larger imitation set.
Forward dynamics gains depend on how outcomes are scored
For forward dynamics, counterfactual supervision reduces RGB prediction error from 14.35 to 9.40 in units of 10^-3 and raises outcome identification from 21.1% to 71.3% among 16 same-state outcomes; at K=2 the same comparison goes from 68.3% to 93.6%. Mixed supervision lands at 9.62 and 67.7% at K=16. Outcome identification asks whether the model picks the realized future that matches by RGB error among several futures starting from the same state, so it measures discrimination between action-dependent outcomes rather than overall image fidelity.
The paper also compares a single multi-objective OpenWAM checkpoint, trained with 60% real joint video-action supervision plus 20% local-context IDM and 20% local-context FDM supervision, against UVA. The unified checkpoint reaches 92.8% policy success versus 88.0% for UVA and has lower IDM and FDM errors on the reported metrics. The authors note two qualifications: its counterfactual dynamics accuracy is weaker than the dedicated specialists, showing a tradeoff between multi-objective training and specialization, and its demonstration-distribution errors are not held-out generalization because some of the same demonstrations may be seen by the policy objective. Its policy score is also below the policy-specialized OpenWAM models, since only 60% of updates optimize the policy.
Composability is shown in simulation, not yet on robots
The framework reports high success on four LIBERO suites and on real-world bimanual tasks, including Toast Bread, Rubik's Cube, and Sort Cups with two Franka FR3 arms. But the transfer study that carries the composability claim is a simulation study: the target-task experiments adapt the video predictor, so they do not establish zero-shot transfer of a complete video-action model, and the authors state that comparable reuse on real robots will require more data-efficient physical dynamics supervision. The FDM study is also limited to local, short-horizon prediction, leaving long-horizon planning and physical deployment as extensions.
The practical priority that follows is specific to this paper's variables. If the goal is a strong policy on a task, the largest measured lever is causal robot-video pretraining, worth roughly 29 to 30 percentage points over the original video checkpoint on LIBERO-Long, with MoT adding about 3 to 5 points on top. If the goal is to reuse a dynamics component after adapting the video predictor, the deciding factor is supervision coverage: a frozen local-context IDM trained with counterfactual data reaches 84.0% mean across four held-out tasks, while the same architecture trained only on demonstrations reaches 21.5%. The choice between full-context and local-context IDMs is not settled by architecture alone; it depends on whether counterfactual transitions are available.
Sources used in this answer
OpenWAM: An Open Framework for Composable World-Action Models
A shared causal robot-video backbone with configurable video-action interaction reaches 97.8% VTA success on LIBERO-Long after robot-video pretraining, and frozen local-context inverse dynamics trained with counterfactual data reaches 84.0% mean success across four held-out LIBERO-90 tasks.