TL;DR

Most AI models "see" an image but then "think" only in words. LanteRn changes this by allowing models to generate latent visual thoughts—high-dimensional vector representations—that interleave with text. By combining Supervised Fine-Tuning (SFT) for grounding and Reinforcement Learning (RL) for utility, LanteRn enables a 3B-parameter model to rival 7B-parameter giants in complex spatial reasoning tasks.

The Problem: The Low-Bandwidth "Textual Bottleneck"

Standard Large Multimodal Models (LMMs) follow a "perceive-then-verbalize" pipeline. They encode an image once and then generate a textual Chain-of-Thought (CoT). This is fundamentally limited because:

1. Information Loss: Converting a rich image into a few sentences destroys fine-grained spatial relationships.
2. Computational Waste: Methods that generate intermediate images (pixel-space reasoning) spend too much energy on "pretty" pixels that don't help solve the logic of the problem.

LanteRn asks: What if the model could maintain a "mental image" in its hidden layers while it reasons?

Methodology: Engineering Latent "Thoughts"

LanteRn augments the transformer architecture with three special control tokens: <|lvr_start|>, <|lvr_sep|>, and <|lvr_end|>.

1. The Supervised Grounding Phase

To prevent the model from generating random noise in the latent space, the authors use the model's own Vision Encoder as a teacher. They extract ROI (Region of Interest) features and force the LLM's hidden states to match these features via a Mean-Squared Error (MSE) loss during SFT.

Figure 1: The LanteRn framework interleaving language with latent visual "thoughts".

2. The RL Alignment Phase

Fidelity to the image isn't everything—utility is. Using Group Relative Policy Optimization (GRPO), the model is rewarded only for the correctness of the final answer. This encourages the model to transform its "mental imagery" from a simple copy of the image into a specialized representation that highlights task-critical information.

Experiments: More with Less

The authors evaluated LanteRn on VisCoT, V*, and Blink. The results show a clear trend: purely textual reasoning (NTP-RL) plateaus, while latent reasoning continues to improve as the model learns to "visualize" its logic.

Table 1: Performance comparison showing LanteRn-RL-8 outperforming the base model and text-only RL baselines.

Key Insights:

• Latent Capacity Matters: Interestingly, performance does not always improve with more latent tokens. There is a "sweet spot" (around K=8 or 16), after which the reasoning chain becomes too diluted.
• The RL "Jump": RL was the "magic ingredient" that allowed the model to use its latent tokens for relational reasoning (Blink-RP), which SFT alone failed to master.

Critical Analysis & Conclusion

LanteRn represents a significant step toward internalized multimodal reasoning. By avoiding the overhead of pixel generation while retaining more detail than text, it finds the "Goldilocks zone" of efficiency.

Limitations:

• Fixed Capacity: The model currently uses a fixed number of latent tokens regardless of the question's difficulty.
• Interpretability: Unlike text CoT, "latent thoughts" are black boxes. We know they work, but we can't easily "read" what the model is imagining yet.

Future Outlook: The next frontier will likely involve dynamic latent reasoning, where the model decides how many "visual thoughts" it needs based on the complexity of the scene. LanteRn proves that for LMMs, the most powerful reasoning might happen in the spaces between words.