Two fixed tokens recover most of RL's math gain
The intervention is deliberately minimal. The authors take a base model, which has only been trained to predict the next token on text and has not been post-trained with reinforcement learning, and they fix the first two tokens of its response. Everything after those tokens is generated normally. The cue supplies no demonstration and no instruction to reason, yet it changes what the model does.
The headline comparison is on MATH-500, a 500-problem test subset of the MATH benchmark, scored by pass@1, meaning the share of problems solved on a single sample. With the cue ".\n\nOkay", Olmo-3-7B rises from 42% to 78% pass@1; its RL-trained counterpart reaches 75%. With "Alright,", Qwen3-14B rises from 72% to 87%, against 86% for its RL counterpart. In these settings the cued base model is not merely improved but competitive with the model that received RL.
The authors also check that the gain is not an artifact of picking one lucky prefix. They search openings with beam search, keeping 20 candidate openings up to two tokens deep and ranking them by the probability averaged across 30 MATH training problems, then sampling 16 responses per problem. They select the opening with the lowest mean answer entropy, a measure of how much the sampled final answers disagree, after discarding prefixes whose missing-answer rate exceeds the no-cue rate by more than 10 percentage points. No prefix was discarded in these runs.
Two cautions belong with this result. Low answer entropy does not imply high accuracy, so the selection rule is a proxy rather than a guarantee. And beam search can miss effective openings that have low probability; when the authors widened the search to 500 two-token openings, the best opening scored 0.784 against 0.773 for the selected cue, and every higher-scoring opening was a variant of "Alright" or consisted of formatting tokens.
Random openings do not reproduce the effect
A natural objection is that any prefix might push the model into a longer, more deliberative response. The control experiments argue against that. Five uniformly sampled token pairs scored between 27.0% and 42.3% pass@1 on Olmo-3-7B under RL-Zero, around the no-cue level of 41.5%, and five random tokens placed after the paragraph break scored 33.3% to 47.1%. All ten random openings produced longer responses and hit the token budget more often than no cue, so length alone does not explain the accuracy gap.
The paragraph break itself is a partial exception worth separating out. The break alone scored 0.766, close to the selected cue's 0.773, which suggests that part of the effect comes from starting a fresh line rather than from the word that follows. Extending the selected cue with additional high-probability tokens added little accuracy and reduced it once the opening became problem-specific.
RL mostly makes the cue more likely
The paper's second finding reframes what reinforcement learning is doing here. RL increases the probability of these cues, and its policy changes concentrate at the first few output positions. In other words, much of the RL gain over the base model can be recovered by fixing the opening the RL model would have produced anyway.
A model-switching experiment makes the asymmetry concrete. On MATH-500 under the RL-Zero prompt, a base model continued by the RL model after the opening reached 43.7% pass@1, close to the base model's own 41.7%. An RL model continued by the base model after the opening reached 77.7%, slightly above the RL model continued by itself at 75.1%. The opening, not the identity of the model writing the rest of the response, carries most of the difference in this setting.
The authors also report that forcing the cue during training rollouts reaches comparable accuracy in fewer RL updates. They support the mechanism with a weight-edit experiment: a single edit to the final MLP output projection, chosen to raise cue likelihood while limiting change on ordinary text, brings accuracy close to the prefilled cue on both Olmo-3-7B and Qwen3-14B without any GRPO updates. The edit barely moves C4 perplexity when the second-moment weighting is used, from 12.890 to 12.895 for Olmo and from 10.651 to 10.651 for Qwen, whereas the unweighted edit raises it by 1.27% and 1.43% respectively.
The cue works because of training-data associations
Why would a word like "Okay" trigger reasoning? The authors' answer is conditioning, in the sense of Pavlov's experiments: tokens that repeatedly precede reasoning-like text in training become associated with the behavior that follows. To test this causally rather than correlationally, they retrain Olmo-3-7B from an intermediate checkpoint on edited data and extend the experiments to SmolLM3-3B.
The intervention is direct. Replacing "okay" with "chicken" in mid-training raises MATH-500 accuracy with the ".\n\nChicken" opening from 2.4% to 37.2%, turning an arbitrary word into an effective reasoning cue. The same logic works in reverse: changing the training associations can remove an existing cue's reasoning effect. The associations also shape how instructions are read. Replacing "step by step" with "duck duck goose" in training makes the prompt instruction "Think duck duck goose" comparably effective at eliciting reasoning to "Think step by step".
A representational check points the same way. The hidden state representations induced by different cues correlate with different document types from the training set. The authors read this as evidence that a cue is not a semantic instruction but a learned pointer to a kind of text the model has seen before.
The claim holds for R1-Zero-style settings
The scope conditions are specific and the authors state them. The analysis focuses on an R1-Zero-style setting, where RL is applied directly to a base model without an explicit post-training stage beforehand. Complex multi-stage post-training pipelines may introduce capabilities beyond those the token cues elicit, so the near-parity result should not be assumed to carry over to them.
Cue effectiveness is also model-dependent, which follows from the account itself: if cues are learned data associations, models trained on different data should have different cues. The authors find no effective cue for Llama-3.1-8B among the openings tested, consistent with prior reports of limited RL gains on Llama and stronger gains after math-focused base-model training. The counterfactual experiments study Olmo and SmolLM3, whose training mixes include synthetic reasoning traces; the cue effects may be especially strong in that setting because those traces repeatedly pair similar openings with reasoning.
Separate what a model can do from what it tends to do
The paper's most useful distinction is between capability and likelihood. If a base model already produces near-RL accuracy once its opening is fixed, then the ability was present before RL; what RL changed was how probable the behavior is. The authors draw out the design question this raises: when a behavior can already be elicited through conditioning, what is gained by making it more likely through weight updates, and which other useful behaviors might become harder to elicit as a result?
For readers evaluating reasoning claims, the practical test is therefore not whether a model reasons after post-training but whether the same model reasons when its opening is controlled. The paper's own evidence sets the comparison: on MATH-500, a two-token cue moved Olmo-3-7B from 42% to 78% and Qwen3-14B from 72% to 87%, while random openings stayed near the no-cue baseline. The authors also note that biases in synthetic training data can amplify cue-driven behavior, and that deliberately pairing useful behaviors with cues during training could make those behaviors easier to elicit, while such associations can also arise unintentionally.
The priority the evidence supports is to treat the opening of a response as a variable to be controlled and reported, not as incidental formatting. Whether a cue helps, does nothing, or backfires depends on the model and on what its training data repeatedly paired with that opening, which is why the same reasoning cue that improves Olmo's math accuracy also makes its refusals more selective while Qwen's math cues do not.
Sources used in this answer
Base Models Can Reason By Taking a Cue From Training Data
Fixing the first two tokens of a base model's response, such as ".\n\nOkay" or "Alright,", raises MATH-500 pass@1 close to RL-trained levels (Olmo-3-7B: 42% to 78% vs 75% for RL; Qwen3-14B: 72% to 87% vs 86%), and causal training-data edits can create or remove these cue effects.