ThinkOmni: Training-Free "Slow Thinking" for the Omni-modal Era

Executive Summary

ThinkOmni is a breakthrough training-free framework designed to bridge the gap between perception-heavy Omni-modal Large Language Models (OLLMs) and the logic-heavy Large Reasoning Models (LRMs). While models like GPT-4o perceive the world well, they often fail at the complex, multi-step logical chain-of-thought (CoT) required for scientific or mathematical tasks.

By treating an off-the-shelf LRM (like DeepSeek-R1 or Qwen3) as a decoding-time guide, ThinkOmni "lifts" textual reasoning into the multimodal space. It requires no fine-tuning and achieves performance gains that rival—and often exceed—models trained with extensive Reinforcement Learning (RL).

The "Thinking" Gap: Why Multi-modality is Hard for Reasoners

Current LRMs (Large Reasoning Models) are primarily text-based. When they encounter an image, audio clip, or video, they are "blind." Conversely, OLLMs are great at saying what is in an image, but they lack the iterative, reflective reasoning (the "slow thinking") needed to solve a physics problem based on that image.

Existing solutions usually involve:

1. Heavy Fine-tuning: Expensive and requires massive amounts of high-quality multimodal CoT data.
2. Captioning: Describing the image and letting a text-LLM solve it. This loses nuanced information and creates a one-way communication bottleneck.

Methodology: LRM-as-a-Guide & Dynamic Scaling

1. LRM-as-a-Guide

The core idea is Logit Mixing. At each step of token generation, the framework calculates three sets of logits:

• Base Logits ($z_{base}$): From the OLLM seeing the full multimodal input.
• Negative Logits ($z^{-}$): From the OLLM seeing only the text prompt (removing the image/audio).
• Positive Logits ($z^{+}$): From the LRM (the "thinker") seeing the text prompt.

The final token distribution is generated by emphasizing the gap between the "Thinker" and the "Blind OLLM," while maintaining the "Seeing OLLM's" grounding.

2. Stepwise Contrastive Scaling

Not every token needs a "Thinker." For example, when the model is just saying "The image shows...", it needs perception. When it says "Therefore, $x=y+2$...", it needs pure reasoning.

ThinkOmni uses Jensen–Shannon (JS) Divergence to measure the disagreement between distributions. If removing the visual input significantly changes the OLLM's output, the model knows perception is vital and lowers the LRM's influence. If the LRM strongly disagrees with the text-only OLLM, the model prioritizes the LRM's reasoning signal.

Performance: Beating the RL-Trained Giants

ThinkOmni was tested across six benchmarks spanning math (MathVista), audio understanding (MMAU), and general omni-reasoning (OmniBench).

Key Results:

• MathVision: Improved Qwen2.5-Omni-7B by 7.9%.
• MMAU (Audio): Achieved 75.5%, outperforming various RFT-based models.
• Efficiency: While it uses three forward passes, the guider (LRM) only processes text, making it significantly faster than other current proxy-tuning methods.

Deep Insight: Visualizing the "Thinking"

The qualitative analysis shows that the LRM's contribution (the guidance weight ${\alpha }_r$) peaks at logical connectives like "Therefore" or "However," while the OLLM's perceptual signals dominate when describing specific entities found in the visual/audio input. This proves the "Fast" vs. "Slow" thinking collaboration is happening naturally.

Conclusion and Limitations

ThinkOmni proves that we don't always need to "retrain" to get smarter. By cleverly mixing the outputs of high-quality "Thinkers" and "Perceivers," we can create a powerful hybrid.

Caveats:

• It requires models to share the same vocabulary (e.g., Qwen-based models).
• It increases inference memory requirements (KV cache for two/three models).

However, as the field moves toward unified "Omni" models, ThinkOmni offers a blueprint for how different specialized modules can collaborate at the speed of thought.