LOTUS: Bridging the Latent-Explicit Gap in LLM Reasoning

TL;DR

Reasoning in Large Language Models (LLMs) has traditionally been a choice between the high accuracy of Explicit Chain-of-Thought (CoT) and the high efficiency of Latent Reasoning. LOTUS (Looped Transformers with parallel supervision on latents) effectively ends this trade-off. By supervising a looped Transformer to "think" in parallel hidden states rather than sequential tokens, LOTUS matches 3B-scale CoT accuracy while delivering up to a 6.9x speedup in thought latency.

The Scalability Wall: Why Does Latent CoT Fail?

Explicit CoT (generating tokens like "Step 1: ...") works because it provides a clear, discrete supervision signal. However, it is slow—every "thought" token requires a full forward pass.

Previous attempts to move this process into the latent space (continuous hidden states) failed at scale (>1B parameters) for two reasons:

1. Iterative Bottleneck: They often generated latents one by one, keeping the sequential delay.
2. Semantic Drift: Without direct grounding, the model's internal "thoughts" became opaque noise that didn't help solve the problem.

Methodology: Looping and Parallel Grounding

LOTUS solves this by re-imagining the reasoning workspace. Instead of adding new tokens, it uses a padded latent prefix—a fixed budget of learnable tokens inserted between the question and the answer.

1. The Looped Architecture

Instead of a standard deep Transformer, LOTUS uses a Looped Transformer. This means the same weights are reused for $R$ iterations. This "recurrent depth" allows the model to refine its internal thoughts over multiple passes without increasing the parameter count.

2. Parallel CoT Supervision

This is the core innovation. During training, the model is forced to align its hidden states $h^{(R)}$ at each latent position with the actual gold CoT tokens using a standard Cross-Entropy loss through the model's own LM head.

Figure 1: (a) The looped forward pass refines latents over R iterations. (b) Parallel supervision aligns latents to reasoning steps via the base LM head.

Revolutionary Results: Efficiency Meets Accuracy

LOTUS is the first latent method to track the "Explicit CoT ceiling" at the 3.2B parameter scale.

• Speed: In natural language reasoning tasks, LOTUS reduced thought-phase latency from 963.6 ms to 140.8 ms (a 6.9x improvement).
• Accuracy: On GSM8K, LOTUS stayed within 1% of explicit CoT, and actually outperformed it on out-of-domain benchmarks like SVAMP.

Figure 2: LOTUS matches the accuracy of explicit CoT across scales while significantly reducing latency.

Interpretable Latents: Looking Inside the Machine

One of the most striking findings of the LOTUS paper is the transparency of the latent space. By projecting the post-loop hidden states back through the LM head, the researchers could "read" the model's thoughts.

Surprisingly, the model didn't just memorize the training paths; it placed high probability on unseen but valid alternative reasoning chains. This suggests that LOTUS learns a genuine manifold of reasoning rather than simple pattern matching.

Critical Analysis & Conclusion

LOTUS represents a significant step toward internalizing inference compute. By moving from discrete, sequential token generation to parallel, continuous state refinement, we can build models that "think" faster without becoming less reliable.

Limitations: The research currently uses a fixed "thought budget" ($K$ blocks, $c$ width). Future work will likely need to make this budget adaptive, allowing the model to loop more for harder questions and less for easier ones—mimicking human cognitive effort.

Takeaway: If you want a model to reason efficiently, don't just hide the thoughts; supervise them in parallel.