TRICE: Bridging the Gap Between Thinking and Executing in Large Reasoning Models
Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
This paper introduces TRICE, a full-pipeline recipe for Teaching thinking models Reasoning interleaved with Integrated Code Execution. By applying this systematic framework to Qwen3 models (4B and 30B), the authors achieve state-of-the-art results among open-source models, notably reaching 96.7% and 99.2% on AIME 2025.
TL;DR
The success of "thinking" models like DeepSeek-R1 and Qwen3-Thinking has proven that long Chain-of-Thought (CoT) is the key to complex reasoning. However, even the best models struggle with "formal" tasks like huge combinatorial searches or precise arithmetic. TRICE (Textual Reasoning Interleaved with Code Execution) provides the full-pipeline recipe to turn these thinkers into "doers." By interleaving natural language with Python execution, TRICE models reach a staggering 99.2% on AIME 2025, outperforming models nearly 10x their size.
The "Delayed-Code" Crisis: Why Thinking Models Underuse Tools
Most current models suffer from a fundamental bias: they treat code as an afterthought. Even when a Python sandbox is available, many models perform 10,000 tokens of fragile mental arithmetic only to call the tool at the very end to "verify" a likely incorrect answer.
The authors identify two core problems:
- Fragile Reasoning Chains: Prompting a strong thinking model with tool-use instructions often degrades its performance (the "text-only" accuracy drops).
- Pattern Improperly Learned: Models often "imitate" the form of tool-use (frequent calls) without understanding the substance (using execution results to pivot the reasoning).
Methodology: The TRICE Recipe
The paper's contribution is not just a model, but a systematic workflow for TIR (Tool-Integrated Reasoning).
1. Data Engineering: The Power of Lightweight Teachers
The authors discovered that teacher selection matters more than teacher accuracy. They compared "High-frequency, Lightweight" snippets (GPT-OSS-120B) against "Low-frequency, Heavyweight" programs (MiniMax-M2.7).
- Insight: Lightweight snippets decompose complex problems into manageable steps that a 4B or 30B model can actually learn to generate and debug.
- Tool-Advantaged Filtering: Only train on problems where tool-use actually outperforms text-only reasoning to ensure the model learns why it's using the tool.
2. The Form-Substance-Noise Progression
The paper provides a fascinating look into the SFT dynamics (see Image Below).
- Stage 1 (Form): The model learns the syntax
tool_codebut fails the logic. - Stage 2 (Substance): Accuracy peaks. The model uses tools to solve sub-problems.
- Stage 3 (Noise): The model starts overfitting to the teacher's idiosyncratic lengths, leading to RL collapse.
Figure 1: Comparison between the "Delayed-Code" baseline (Left) and the TRICE interleaved pattern (Right).
3. Stabilizing RL for Tool Use
Training for TIR via Reinforcement Learning is notoriously unstable because the feedback from the sandbox (tool returns) is "off-policy" to the language model's weights.
- Solution: Fully on-policy training combined with Rollout Routing Replay. This ensures the model doesn't "hack" the reward or collapse during multi-turn interactions.
Experiments: SOTA Achievement
TRICE models (4B and 30B) were tested on five competition-level math benchmarks.
Key Takeaway: TRICE-30B dominates the ~30B scale, significantly beating GLM-4.7-Flash and Nemotron-3-Nano.
Deep Insight: Code as a Cognitive Tool
The authors didn't just look at accuracy; they analyzed what the code does. They categorized tool use into:
- Empirical Discovery: Testing small cases to find a pattern.
- Algorithmic Search: Brute-forcing combinatorial spaces.
- Computation Offloading: Arithmetic and symbolic algebra.
- Conjecture Verification: Checking intermediate steps.
Discovery and Search are the biggest winners. TIR allows a 30B model to solve problems that a 235B model cannot, simply because some math problems require "searching" an area too wide for text-only simulation.
Conclusion & Limitations
TRICE proves that for "thinking" models, the tool is not just a calculator—it's an extension of the working memory and the "policy head." By following a rigorous pipeline of tool-advantaged SFT and stabilized RL, the reasoning frontier can be pushed much further without exponentially increasing parameter counts.
Limitations: The current recipe is math-heavy. Future work is needed to see if this "interleaved" pattern transfers to more subjective agentic tasks like web browsing or software engineering where "verifiable rewards" are harder to define.