DR-Venus: Scaling Deep Research Agents to the Edge with 4B Parameters

Executive Summary

TL;DR: DR-Venus is a breakthrough 4B-parameter agent that proves edge-scale models can perform "Deep Research"—complex, multi-step information seeking—traditionally reserved for giant 30B+ models. By refining just 10K open-data trajectories through a two-stage SFT and Reinforcement Learning (RL) pipeline, DR-Venus sets new SOTA benchmarks for small models.

Positioning: This work is a SOTA-shattering recipe for small language models (SLMs). It shifts the focus from "scaling laws" (bigger is better) to "data utilization laws," showing that dense supervision can make a 4B model punch significantly above its weight class.

Problem & Motivation: The "Sparse Reward" Trap

Building a research agent is harder than building a chatbot. An agent must plan, search, browse, and synthesize over 100+ turns. For small models, two major obstacles exist:

1. Sensitivity to Noise: Small models are easily "confused" by messy tool-use formats or redundant search steps in training data.
2. RL Collapse: In traditional RL (like GRPO), if an agent fails to find the right answer in a 200-turn rollout, the reward is zero. For a 4B model, successful "lucky" rollouts are rare, leaving the model with nothing to learn from—a phenomenon known as advantage collapse.

Methodology: The Secret Sauce of DR-Venus

The Venus team at Ant Group solved this using a two-stage refinement process.

1. Agentic SFT with Turn-Aware Resampling

Instead of just copying any open-data trajectory, they cleaned the REDSearcher dataset and applied Turn-Aware Resampling. They upweighted longer, more complex trajectories (100+ turns) by 5x. This forced the 4B model to learn sustained planning rather than just "quick-fire" answering.

2. IGPO: Turn-Level Information Gain

The core innovation is Information Gain-based Policy Optimization (IGPO). Instead of waiting for a final "Correct/Incorrect" reward at the end of 200 turns, the model gets a reward at every single turn.

• The Intuition: If a browsing step makes the model more confident about the final ground-truth answer, that step receives a positive reward.
• Format-Aware Penalty: To fix the common issue of SLMs "breaking" XML tags, the researchers applied a precise penalty only to the malformed turns, rather than the whole trajectory.

Figure 1: DR-Venus-4B-RL significantly leads the <9B parameter category.

Experiments: Narrowing the Scale Gap

The results on BrowseComp and GAIA benchmarks are startling. DR-Venus-4B doesn't just beat other 4B models; it beats many 30B models (like OpenResearcher-30B) and approaches proprietary giants.

Key Insight: The Browse Ratio

A fascinating finding in the ablation studies was the Browse Ratio. Successful trajectories consistently showed a higher ratio of "Browsing" (reading pages) versus just "Searching" (viewing snippets). RL effectively "taught" the 4B model that deep reading produces better answers than shallow searching.

Figure 2: Successful trajectories correlate with higher browsing activity.

Deep Insight: The Latent Potential of SLMs

The paper's most provocative finding comes from Pass@K analysis. While the Pass@1 (first try) of a 4B model is modest, its Pass@16 (success in 16 tries) is elite. On BrowseComp-ZH, DR-Venus-4B-SFT hit 78.5%, outperforming GPT-5 High and Gemini-3-Pro.

This suggests that the "intelligence" is already there in the 4B model; RL's job isn't necessarily to increase the maximum capability, but to improve the reliability so the model hits the right answer on the first try.

Critical Analysis & Conclusion

Takeaway: DR-Venus proves that edge-scale agents are no longer "toys." By using dense rewards (IGPO) and high-quality trajectory resampling, developers can deploy local, private, and cheap research assistants that rival cloud-based giants.

Limitations: The model still relies on an external LLM (Qwen3-30B) for webpage summarization within the tool server. Future work should aim to consolidate this summarization task into the 4B agent itself to achieve true edge autonomy.

DR-Venus marks a shift toward Test-Time Scaling—where we focus on how the model "thinks" during the task rather than just how many weights it has.