GRAM: Beyond Determinism – Unlocking Multi-Trajectory Probabilistic Reasoning

TL;DR

Generative Recursive reAsoning Models (GRAM) transform the traditionally deterministic process of latent state refinement into a stochastic generative journey. By treating reasoning as a probabilistic trajectory, GRAM allows models to explore multiple hypotheses in parallel, effectively outperforming much larger models and deterministic recursive baselines on complex puzzles and multi-solution tasks.

Context: Why "Deep" Isn't Enough

In the quest for efficient neural reasoning, Recursive Reasoning Models (RRMs) have emerged as a compact alternative to massive autoregressive LLMs. Tools like HRM and TRM use a fixed set of weights to iteratively refine a hidden representation—effectively scaling "depth" without blowing up parameter counts.

However, these models have a critical flaw: Determinism. In a deterministic RRM, the path from problem to solution is a single line. If the model gets stuck in a local minimum or a "reasoning trap," there is no escape. This is particularly problematic for tasks with multiple valid solutions (like N-Queens or Graph Coloring) where the model inevitably suffers from mode collapse.

Methodology: The Architecture of Stochastic Guidance

GRAM redefines reasoning as a stochastic latent trajectory. Instead of a fixed update, the model samples a transition at every step.

1. The Hierarchical Loop

GRAM splits the latent state $z$ into two parts:

• High-level ($h$): Carries abstract reasoning state and is updated stochastically.
• Low-level ($l$): Handles fine-grained intermediate computation and updates deterministically at a faster frequency.

2. Stochastic Guidance

The core innovation is the high-level transition: $z_t=u_t+{ϵ}_t$ where $u_t$ is a deterministic proposal and ${ϵ}_t$ is a Stochastic Guidance signal sampled from a learned Gaussian distribution. This allows the model to "explore" the reasoning space during inference.

Figure 1: The GRAM architecture showing the interaction between deterministic low-level refinement and the stochastic high-level guidance.

3. Training via Variational Inference

To train this, the authors use Amortized Variational Inference. They optimize an Evidence Lower Bound (ELBO), where a variational posterior (which "sees" the answer during training) teaches the prior (which only sees the question) how to navigate the latent space.

Experiments: Scaling along the "Width" Axis

GRAM introduces a new dimension for Test-Time Compute: Width. While standard models can only think "longer" (more iterations), GRAM can think "wider" by sampling multiple trajectories in parallel.

SOTA Results on Puzzles

On Sudoku-Extreme, GRAM hits 97% accuracy, crushing the 87.4% of TRM. Even more impressive is its performance on the ARC-AGI benchmark, where it significantly narrows the gap between compact recursive models and human-level abstract reasoning.

Table 1: GRAM vs. Baselines on multi-solution tasks.

Overcoming Mode Collapse

In multi-solution tasks (N-Queens), deterministic models fail as the number of possible solutions grows. GRAM, however, maintains consistent accuracy because its stochastic nature lets it discover different valid outcomes independently.

Figure 2: Scaling compute. Notice how GRAM (N=20 parallel samples) achieves higher accuracy with 16 iterations than a deterministic model (TRM) does with 320 iterations.

Deep Insight: Training the Prior

The magic of GRAM isn't just "adding noise." The ablation studies show that Naive randomness (sampling from the decoder or random $z_0$) actually hurts performance. The benefit comes specifically from the Variational framework, where the model learns a structured prior that knows how to be uncertain and how to explore.

Conclusion & Limitatons

GRAM proves that reasoning is better modeled as a probability distribution than a deterministic function. It provides a blueprint for "Width-based" scaling which is naturally parallelizable on modern GPUs.

Limitations: The primary bottleneck remains the sequential nature of recursive training (Deep Supervision), which is currently less efficient than the parallel training of standard Transformers. Scaling this to "Foundation Model" size remains the next big challenge.

Final Takeaway: Future reasoning systems should not just aim for deeper thought, but more diverse thought.