Can a reasoning recipe improve zero-shot robot foundation model performance?

Reasoning supervision instead of more robot data

The paper asks whether an existing robot foundation model can be made better without enlarging it, collecting more demonstrations, or repeating foundation-scale training. Its answer is to change what the model is trained to condition on: instead of only an instruction and observations, the policy also receives a short natural-language account of the current situation, why the next action is appropriate, and what that action should accomplish. The authors call this recipe ARC.

The claim is deliberately narrow. ARC does not alter the model architecture or add a new robot dataset; it relabels demonstrations that already exist and then fine-tunes the pretrained model to use those labels for control. The reported gains are therefore attributed to the reasoning content and the training procedure built around it, not to extra robot experience.

Traces must be grounded in the next action

The central design choice is what a reasoning trace should contain. The authors argue that useful traces are both causal and action-grounded: they explain why an action fits the current state and what effect it should produce, while also naming the concrete step the robot should take. Each annotation in their scheme links a perceived state and relevant cause to a possible consequence, a chosen effect, an action implication, a forbidden effect, and a completion status.

This matters because the trace is not a free-form commentary on the scene. It is aligned to an action chunk, so the language is tied to the same observation and the same segment of robot behavior that the policy must predict. The paper's ablation supports the distinction: on RoboLab-120, the full six-component trace reached 45.3% success, while an action-only trace reached 27.42% and a state-plus-action trace reached 32.25%, both below the 28.0% base policy only in the action-only case and still far from the full trace.

Labels come from existing demonstrations

Robot demonstrations record observations and actions but usually not the reasoning behind each decision. ARC-Trace addresses that gap with a two-stage automatic labeling pipeline. Gemini 2.5 Pro watches an episode video and produces a temporally grounded narrative while selecting important frames; GPT-5.5 then generates episode-specific state checks and combines them with the task instruction, narrative, and wrist observation into a structured trace.

The scale is concrete. The pipeline was applied to the DROID v1.0.1 training split, which contains 95,658 episodes. It labeled approximately 74,740 episodes, or 78.1% of the split, producing about 1.2 million annotated frames, roughly 16.1 annotations per labeled episode. Most unlabeled episodes lacked a usable task instruction. The authors note that the state checks are model-generated descriptions of visual evidence, not independently measured ground-truth states, and the full episode video is used only offline to build supervision, not as an input to the deployed policy.

One trace set, two model architectures

ARC is tested on two different kinds of robot foundation model: π0.5, a vision-language-action model with a PaliGemma backbone and a separate 300M-parameter action expert, and Cosmos3-Nano-Policy, a world-action model that pairs a vision-language reasoner with a diffusion generator. The same traces and training data are used across both, but fine-tuning and inference are tailored to each architecture.

At inference the fine-tuned models are frozen, and an external vision-language model generates traces from the task instruction and current camera observations, refreshing them during execution. The authors chose external generation because the models' built-in language components were less reliable at producing these traces than at using them for control. They report that performance stays comparable across the external VLMs evaluated, and that a near-real-time VLA deployment added about 10 ms of control-loop overhead.

Gains vary by benchmark and model

The headline numbers need their benchmarks kept separate. On MolmoSpaces, task success improved by 27.2 percentage points for π0.5 and 18.6 percentage points for Cosmos3-Nano-Policy. On the reasoning-focused RoboLab-Reasoning-50, the gains were 46.8 percentage points for π0.5 and 50.0 percentage points for Cosmos3-Nano-Policy. On RoboLab-120, ARC improved both models across all three settings, with the adapted models ranking first and second in each setting.

These are zero-shot results relative to each model's own baseline, not comparisons against a newly trained model of larger scale. The paper also reports efficiency side effects: Cosmos3-Nano-Policy reached baseline success with 4.3 times fewer training updates, and π0.5 and Cosmos3-Nano-Policy matched their base policies at inference with 10 times and 2 times fewer denoising steps respectively.

Real-robot success rises most on hard tasks

The largest reported gain is on physical hardware. On real robots, ARC improved π0.5's task success by 82.2 percentage points, with the authors attributing the improvement to reasoning and long-horizon tasks. The paper also reports that ARC-trained models more reliably followed underspecified instructions, inferred missing steps, completed long-horizon tasks, adapted to changes, and recovered from mistakes.

That 82.2-point figure should not be read as a general success rate for all robot tasks. It is a change measured on the paper's real-robot evaluation of reasoning and long-horizon tasks, and the abstract presents it as an improvement over the corresponding baseline rather than as an absolute level. The simulation gains and the hardware gain answer related but distinct questions.

The recipe helps most when reasoning is the bottleneck

The pattern across results is that ARC's advantage grows as tasks demand more inference. The MolmoSpaces gains are in the tens of points, the reasoning-focused RoboLab-Reasoning-50 gains approach 50 points, and the real-robot reasoning and long-horizon evaluation shows the largest jump. The authors' interpretation is that causal supervision makes each demonstration more informative, because the model receives the reason for an action alongside the action itself.

The ablations sharpen this. Removing the action implication dropped RoboLab-120 success to 30.58%, and removing cause and effect dropped it to 39.25%, both below the 45.3% full-trace result. Full fine-tuning also mattered: π0.5 reached 45.0% with full fine-tuning versus 30.9% with action-head-only training, and Cosmos3-Nano-Policy reached 51.2% versus 43.7% with generator-head-only training. The authors read this as evidence that pretrained language features must be reshaped, not merely read out, for the controller to use causal distinctions.

What the recipe leaves open

ARC depends on an external vision-language model to generate traces at inference, so its behavior is tied to that generator and to the refresh schedule used during execution. The paper reports comparable performance across the external VLMs it evaluated, but it does not claim that any generator will work. The labeling pipeline also inherits the limits of its source data: about a fifth of the DROID training split went unlabeled, mostly for lack of a usable task instruction.

The authors list policies that reliably generate their own causal traces and extension across embodiments as future directions, which places both outside what this paper demonstrates. Within the evidence presented, the practical priority is clear: when a pretrained robot foundation model already has broad manipulation competence but struggles on reasoning-heavy, underspecified, or long-horizon tasks, adding action-grounded causal traces and fine-tuning the full model is the intervention this paper supports, and the size of the expected gain should be read against the specific benchmark or hardware setting rather than as a single universal number.

Sources used in this answer