[NVIDIA] Nemotron-Cascade 2: Achieving Gold-Medal Reasoning with 3B Activated Parameters

TL;DR

NVIDIA has released Nemotron-Cascade 2, an Open 30B Mixture-of-Experts (MoE) model. Despite being significantly smaller than "frontier" models (only 3B parameters activated per token), it has officially hit Gold Medal-level performance in the 2025 International Mathematical Olympiad (IMO) and International Olympiad in Informatics (IOI). The secret sauce? A refined Cascade RL pipeline and a novel stabilization technique called Multi-Domain On-Policy Distillation (MOPD).

The Problem: The "Reasoning-Alignment" Seesaw

In LLM post-training, researchers usually face a trade-off. If you optimize heavily for reasoning (e.g., math and code via RLVR), you often see a drop in instruction following or general helpfulness. Conversely, RLHF for human alignment can "soften" a model’s logical rigor.

Traditional joint training attempts to balance these by mixing data, but as the number of domains grows, the reward signals become conflicting and the training becomes unstable.

Methodology: The Architecture of a Cascade

Nemotron-Cascade 2 moves away from "one-size-fits-all" RL. Instead, it treats post-training as a carefully orchestrated sequence.

1. Sequential Domain-Wise RL

The authors identified that the order of training matters immensely. They start with Instruction-Following RL (IF-RL) to set a foundation of adherence, then move into STEM and Tool-use RL. This builds "intelligence density" before the model is even introduced to human preference data.

2. Multi-Domain On-Policy Distillation (MOPD)

This is the technical highlight of the paper. To stop the model from "forgetting" how to be a good math solver while learning how to be a helpful assistant, the authors use MOPD.

• Teachers from Within: They take the best intermediate checkpoints from earlier stages as "domain teachers."
• Dense Signal: Unlike GRPO which uses sparse, sequence-level rewards, MOPD provides a dense, token-level distillation advantage. It effectively "re-anchors" the model to its peak performance in specialized domains.

Figure: The sequential flow from SFT to IF-RL, Multi-domain RL, MOPD, and finally SWE agent training.

Experiments & Results: Punching Above Its Weight

The most striking result is the model's performance on Olympiad-level problems. In IMO 2025, it solved 5 out of 6 problems, scoring 35/42—a Gold Medal standard.

SOTA Comparison

In the coding arena, Nemotron-Cascade 2 (at 30B) matches or exceeds the performance of models 20x its size:

• ArenaHard v2: 83.5 (Beating Qwen3.5-35B at 65.4).
• LiveCodeBench v6: 87.2 (Compared to 78.7 for the 120B Nemotron-3-Super).
• IOI 2025: 439.28 (A Gold Medal score).

Table: Comparison across LiveCodeBench and Codeforces ELO ratings showing superiority over 100B+ parameter baselines.

Deep Insight: Why Small MoE Wins

The paper proves that "Intelligence Density" is a more critical metric than raw parameter count. By using MoE, NVIDIA keeps inference efficient (3B active parameters), while the Cascade RL ensures that those parameters are utilized with specialized expertise.

The use of Tool-Integrated Reasoning (TIR)—allowing the model to call a Python executor during the "thinking" phase—acts as a massive force multiplier for hard math and code problems.

Conclusion & Limitations

Nemotron-Cascade 2 is a milestone for open-weights models. However, the authors noted it still lags behind in knowledge-intensive benchmarks compared to Qwen3.5, suggesting that while RL can maximize reasoning, the "base" knowledge is still capped by the pre-training quality.

The Takeaway: If you are building reasoning agents, don't just throw all your RL data into a single pot. Segment them, sequence them, and use your own best intermediate models as teachers to stabilize the climb to SOTA.