Tongyi DeepResearch: Pioneering the Era of Open-Source AI Researchers
Tongyi DeepResearch Technical Report
Tongyi DeepResearch is an agentic large language model (30.5B total, 3.3B activated) designed for long-horizon information-seeking tasks. Using an end-to-end training framework of agentic mid-training and reinforcement learning, it achieves SOTA performance on benchmarks like Humanity’s Last Exam and BrowseComp, outperforming OpenAI o3 and DeepSeek-V3.1.
TL;DR
Alibaba's Tongyi Lab has released Tongyi DeepResearch, an agentic MoE model (30.5B total/3.3B active parameters) that sets a new SOTA for autonomous online research. By moving beyond traditional SFT+RL and introducing Agentic Mid-training and Markovian Context Management, this model solves the "context suffocation" problem and performs multi-step discovery at a PhD level.
The "Agentic Gap" in Foundation Models
Most LLMs today are "accidental agents." They are trained on static web text and then asked to perform dynamic research via prompting. This creates two major bottlenecks:
- Inductive Bias Conflict: Pre-training on plain text doesn't teach a model how to handle environment feedback (Observations).
- Context Suffocation: In long-horizon tasks (30+ steps), the attention window becomes filled with irrelevant search results, causing the model to lose track of its original goal.
Methodology: The End-to-End Agentic Pipeline
Tongyi DeepResearch introduces a three-tier design to bridge these gaps:
1. Agentic Mid-training (The Bridge)
Unlike traditional pipelines that go straight from Pre-training to SFT, the authors insert a Mid-training phase. This involves training the model on large-scale synthetic agent trajectories. This "Agentic CPT" (Continual Pre-training) embeds the ability to plan and reflect directly into the model's weights.
2. Markovian Context Management
To handle 100+ tool interactions without breaking the 128K context window, the model uses a dynamic workspace reconstruction. instead of feeding the entire history, it provides:
- The original Question.
- (Condensed Memory): An evolving report that summarizes past findings.
- The latest Observation.
3. Automated PhD-Level Data Synthesis
The team solved the data scarcity problem by using LLMs to generate "Super-human" questions. They use set theory and "Uncertainty Injection" to create complex, multi-hop queries that would take a human researcher hours to answer.
Experiments: Efficiency Meets Power
Tongyi DeepResearch was tested against the giants: OpenAI o3, DeepSeek-V3.1, and Gemini Deep Research.
| Benchmark | Tongyi DeepResearch | OpenAI o3 | DeepSeek-V3.1 |
|---|---|---|---|
| Humanity's Last Exam | 32.9 | 24.9 | 29.8 |
| FRAMES | 90.6 | 84.0 | 83.7 |
| BrowseComp | 43.4 | 49.7 | 30.0 |
The model's Heavy Mode utilizes test-time scaling—launching parallel agents and synthesizing their findings—which boosted scores on BrowseComp-ZH to 58.1%, demonstrating that more "interaction turns" are as valuable as "thinking tokens."
Critical Insight: The "Wind Tunnel" Effect
The authors highlight a crucial finding: Reinforcement Learning (RL) performance depends more on environment stability than the algorithm itself. They built a "Simulated Wiki" environment (an offline RAG sandbox) to act as a "wind tunnel" for testing RL strategies before deploying them to the noisy, real-world internet.
Conclusion and Future Outlook
Tongyi DeepResearch proves that 3B active parameters are enough to outperform 100B+ parameter models if the agentic training is done correctly. By open-sourcing the model and the data synthesis framework, Alibaba is democratizing the ability to build "AI Researchers."
The next frontier? Moving from domain-specific research agents to General-purpose Foundation Agents that can operate across any UI or toolset with minimal supervision.