Can Structured Reasoning Improve Long-Horizon Textual World Modeling?

Recursive rollout lets intermediate errors compound

A world model has to predict how an environment evolves under a sequence of actions so an agent can compare possible futures before acting. The common recipe is to learn a one-step transition and roll it forward, feeding each predicted state back in to produce the next. That factorization is convenient and exposes intermediate states, but every intermediate error becomes part of the context for all later transitions, so errors can compound with horizon.

The paper's alternative is non-recursive prediction: condition on the current state plus the whole future action sequence and predict the target state directly. The stated difficulty is that long-horizon prediction is inherently multi-horizon — one direct model covering many horizons is hard to train, while separate per-horizon models fragment the data and block shared learning. This matters most in partially observable environments, where the model must preserve history while tracking action-dependent changes over many steps.

Sparse deltas replace full state rewrites

The proposed transition, latent-SR, treats the multi-step evolution as one structured trajectory. Each action induces a sparse change to an otherwise persistent state, and the model emits those sparse deltas in a single autoregressive decode, applied to the persistent state to obtain the intermediate states without re-entering the model. Because the edits are never fed back as inputs to a new transition call, they can be supervised without reintroducing recursive feedback.

The paper frames this against direct endpoint prediction the way an immediate answer contrasts with reasoning through a chain of thought — except the intermediate reasoning here is an explicit, semantically grounded world-state trajectory that can itself be supervised and rewarded. A supporting ablation on a shared 500-example ScienceWorld subset compares sparse delta reasoning against full-state reasoning that regenerates the complete intermediate state after each action: delta reasoning raises Fact-F1 from 0.499 to 0.556 with Qwen3-4B and from 0.597 to 0.625 with GPT-5.2. Free-form thinking, using unstructured chain-of-thought text instead of a state-schema-constrained format, scores lower on the same subset with Qwen3-4B (0.465) and GPT-5.2 (0.606).

Predictive gain and mid-trajectory rewards do different jobs

Two training signals are central. A predictive-gain objective rewards the learned state for improving over a matched predictor that conditions on raw history instead. Intermediate predictive rewards then supervise each state along the trajectory, which matters because endpoint supervision alone provides weak credit assignment and intermediate predictions can stay plausible while losing information needed for later states.

The CEO-Bench ablation separates these components. Starting from distillation at 83.49 average SE-score, trajectory RL raises the average to 84.86 when initialized from distillation, whereas RL without that warm start drops to 61.12 — teacher-distilled latent trajectories are an important initialization in this setting. Adding predictive gain lifts the average to 86.58, and adding intermediate-state rewards further improves it to 87.35. The paper reports that discrimination is null (median approximately 0, non-significant) for every latent-SR variant prior to intermediate-state supervision, attributing the full model's significant positive margin specifically to that reward component rather than to distillation, trajectory RL, or predictive gain alone.

Gains grow as the prediction horizon lengthens

Across ScienceWorld, Jericho, and CEO-Bench, latent-SR achieves the strongest average performance on every metric, and the paper states the gains are concentrated at longer horizons. At h10 it exceeds obs-direct by nearly 10 Fact-F1 points and 13 BLEU points on ScienceWorld, and by about 7 SE-score points on CEO-Bench.

The per-horizon tables show the shape of that advantage. On CEO-Bench, latent-SR scores 93.76 at h1 and 81.73 at h10, averaging 87.35, against 81.85 for obs-direct and 81.08 for obs-recursive. On ScienceWorld, latent-SR averages 92.44 Fact-F1 and 92.37 BLEU, versus 88.94 and 85.92 for obs-direct. Jericho is the exception worth noting: obs-recursive leads at the shortest horizons (h1, h5) on both BLEU and Score MAE, but latent-SR overtakes it from h10 onward and attains the best average on both metrics (24.5 BLEU, 1.728 Score MAE).

The paper also separates the full-system gap. latent-direct — same representation and training through predictive gain, but a direct transition — reaches 89.16/87.63 Fact-F1/BLEU on ScienceWorld and 83.53 SE-score on CEO-Bench, ahead of obs-direct but well below latent-SR. At the matched predictive-gain stage, latent-SR's own ablations reach 86.58 SE-score on CEO-Bench and 91.91/91.10 on ScienceWorld, so the transition architecture accounts for most of the gap over latent-direct, with the mid reward contributing a smaller remainder.

Injected false facts persist under recursion but fade under latent-SR

To probe error propagation directly, the paper intervenes on the encoded state in CEO-Bench while holding the future action sequence fixed, injecting the same fabricated fact across architectures and reporting the SE-score drop under the fixed-target protocol. The relevant perturbation inserts a fabricated −$5,000/week exogenous cash shock contradicting the realized trajectory; the irrelevant condition inserts a latency distractor outside the four scored fields.

The relevant perturbation harms every model while the irrelevant distractor has near-zero effect, which the paper reads as evidence the intervention is selective rather than generic corruption. For latent-recursive, the relevant perturbation causes an 8–14 point SE-score loss across horizons, averaging 11.1. The average loss is 5.4 for latent-SR after distillation and 1.7 for the full model. The paper's explanation is that latent-recursive repeatedly turns predicted intermediate states into inputs to later transition calls, so an erroneous fact can persist, whereas latent-SR conditions a single multi-step generation on the original encoded state without recursively re-entering reconstructed states. The authors state this does not by itself identify the internal mechanism but is consistent with reduced propagation through recursive feedback.

Trajectory RL further improves robustness while clean performance rises too: holding the latent-SR architecture fixed, mean perturbation damage falls from 5.4 after distillation to 1.7 in the full model, while clean SE-score increases from 83.2 to 87.1 on the same subset. The paper notes the robustness gain is therefore not obtained by simply degrading clean prediction quality.

Only latent-SR shows significant sensitivity to future actions

Aggregate accuracy does not show whether a model's predictions actually track the supplied action sequence. The paper tests this on CEO-Bench by forking a shared history into three branches — aggressive growth, cost cutting, and a resample control — that diverge by horizon 10, then asking each model to predict one branch's outcome given only that branch's actions.

In the company-clustered counterfactual analysis, latent-SR is the only model with statistically significant sensitivity to future actions. obs-recursive and obs-direct are significantly negative under the clustered sign test. The paper explains that their permutation p = 1.000 is not interpreted as evidence against action sensitivity, because the median-of-medians statistic degenerates when a majority of per-company medians tie at exactly the same value, most commonly zero; a mean-based statistic does not share that failure mode.

A concrete branching case study illustrates what the adjusted margin measures. One source company ($811K cash, 139 customers at the branch point) forks into three directives. All three branches share the same first-week actions, so every method's h=1 cash forecast is nearly identical; the branches diverge sharply by h=10. latent-SR tracks the divergence in the correct direction, staying within $1.5K–$1.7K of ground truth on the two flatter branches and within $345K on the sharply diverging aggressive-growth branch. Chronos-2, which does not condition on branch-specific future actions, produces the same $815K–$850K trajectory across all three branches and misses the true aggressive-growth outcome by $599K at h=10.

The paper also runs a within-model action-permutation test that needs no external reference model: it compares a prediction conditioned on a branch's own actions against one conditioned on a different branch's actions from the same starting state, scored against the original branch's ground truth. The authors note that a majority of companies have an exact-zero delta for a sizable fraction of checkpoints and horizons — the model produces a scoring-identical prediction regardless of which branch's actions it is given — which is why they aggregate to a per-company mean rather than a median.

Where these results stop applying

The experiments focus on textual, partially observable simulated environments: ScienceWorld, Jericho, and CEO-Bench. CEO-Bench is a synthetic business-simulation environment and does not involve real companies, individuals, or financial data. Whether the same factorization benefits visual or continuous world models, or improves downstream closed-loop planning, remains open in the paper's own framing.

Two further boundaries sit next to the results they qualify. First, the counterfactual sensitivity evidence rests on CEO-Bench alone, and the paper's own clustered analysis documents a degeneracy in the median-based permutation statistic that forces reliance on sign tests for the two raw-history baselines. Second, the paired CEO examples show the advantage is aggregate rather than uniform: in one case latent-SR better captures subscriber and issue decline but its cash forecast is less accurate than obs-direct's, and in another it underestimates cash and open issues even while preserving the direction of subscriber growth. The paper also notes that in one near-depletion case latent-SR's unscored cancellation forecast was 509 against a ground truth of 16, so that example is not evidence of accurate flow prediction throughout the window.

Judge a transition model by horizon slope and action sensitivity

The decision-relevant split in this paper is not which model wins on average but how the gap behaves along two axes. The first is horizon slope: latent-SR's advantage over obs-direct widens from h1 to h10 on ScienceWorld and CEO-Bench, while on Jericho obs-recursive still leads at h1 and h5 and only loses the lead from h10 onward. A reader comparing transition designs should therefore check the per-horizon curve rather than the average, because an average can hide a crossover.

The second axis is whether predictions respond to the action sequence at all. The within-model permutation test is the sharpest diagnostic here: it asks whether a model's own prediction changes appropriately when only the conditioning actions change, holding the starting state fixed, and the paper reports that a majority of companies show an exact-zero delta for a sizable fraction of checkpoints and horizons. That is a concrete failure signature to look for, and it is distinct from aggregate accuracy — a model can score well on average while producing the same forecast regardless of which branch's actions it is given.

The paper's own priority ordering follows from its ablations: the transition architecture accounts for most of the gap over latent-direct, with the intermediate-state reward contributing a smaller remainder, and the intermediate-state reward is specifically what turns a null counterfactual margin into a significant positive one. For anyone deciding where to spend effort on a long-horizon textual world model, that ordering — non-recursive structured transition first, intermediate-state supervision second — is the paper's most transferable claim within the environments it tested.

Sources used in this answer