Parallel Test-Time Scaling: Bringing the Power of "Search" to Latent Reasoning

TL;DR

Test-time scaling (TTS)—the ability to boost model performance by spending more compute at inference—has been a "token-only" luxury. This paper breaks that barrier by enabling parallel TTS for Latent Reasoning Models. By introducing stochasticity into continuous vector spaces and training a dedicated Latent Reward Model (LatentRM), the authors show that models can explore multiple "silent" reasoning paths and pick the best one, achieving higher accuracy with far greater efficiency than standard natural language reasoning.

Problem & Motivation: The "Probability Gap" in Continuous Space

In standard LLMs, we scale compute by sampling multiple text-based chains (e.g., Self-Consistency). This works because:

1. Sampling is easy: We have a vocabulary and logits (probabilities) to pick from (Top-p, Temperature).
2. Aggregation is clear: We can use majority voting on the final text string.

However, Latent Reasoning Models (like COCONUT or CoLaR) replace text tokens with hidden vector representations. These vectors don't have a "vocabulary" or inherent probability scores. This creates two major roadblocks:

• How do you "sample" a vector? Without a probability distribution, it's unclear how to generate diverse paths.
• How do you "rank" a vector? There is no linguistic form for a Reward Model or human to judge.

Methodology: Sampling and Scoring the Invisible

The authors propose a dual-component framework to address these challenges.

1. Stochastic Latent Sampling

Instead of picking tokens, the model injects "uncertainty" into the latent space using two methods:

• Monte Carlo Dropout (MC-Dropout): Keeps dropout active during inference to sample different sub-networks, capturing the model's "internal doubt" (epistemic uncertainty).
• Additive Gaussian Noise (AGN): Adds random isotropic noise to the latent vectors, simulating "input ambiguity" (aleatoric uncertainty).

2. The Latent Reward Model (LatentRM)

To pick the winner among "silent" trajectories, the authors trained a LatentRM.

• The Architecture: A scoring head attached to the latent backbone.
• The Innovation: Instead of simple binary labels (Right/Wrong), they use Step-wise Contrastive Supervision. This teaches the model to distinguish which latent "thought" at step t is more likely to lead to a correct answer compared to its peers.

Experiments & Results: Efficient Intelligence

The framework was tested on several backbones including Llama-3.2-1B and RoT-4B.

Key Findings:

• Scaling Success: Accuracy increases monotonically with the number of samples ($N$). More compute consistently equals more "intelligence."
• Efficiency Reign: Latent reasoning with TTS (N=4) matches the accuracy of explicit CoT but is roughly 5x faster in terms of decoding steps.
• Exploration Dynamics: MC-Dropout tends to find "harder" solutions by drifting further from the deterministic path, while AGN provides a "firework" pattern of diversity around the center.

Deep Insight: Beyond Linguistic Limits

The most profound takeaway is that reasoning does not require language. By scaling in the latent space, we bypass the "verbosity" tax of traditional CoT. The LatentRM acts as a "gut feeling" for the model, allowing it to navigate a continuous manifold of thoughts toward the right answer.

Limitations & Future Work

• Model Scale: Most experiments were on 1B-4B models. The behavior of these "latent landscapes" in 70B+ models remains an open question.
• RL Integration: The next step is clearly to integrate this into a Reinforcement Learning loop (like PPO or DPO) where the model learns to optimize its own latent trajectories for better searchability.

Conclusion: This work proves that the "Search" paradigm—central to the success of models like OpenAI's o1—can be successfully ported to the silent, efficient world of latent representations.