[EMNLP 2025] STACK: Breaking the Overthinking Cycle in Large Reasoning Models

TL;DR

Large Reasoning Models (LRMs) like DeepSeek-R1 have revolutionized complex problem-solving via long Chain-of-Thought (CoT). However, they often "overthink," wasting tokens on repetitive verification. STACK (State-Aware Reasoning Compression with Knowledge Guidance) introduces a surgical approach to this problem: it monitors the model's internal entropy to detect hesitation, uses RAG to correct baised paths, and applies a reward-driven training strategy to maximize "Token Efficiency."

The "Overthinking" Tax: Why Long CoT is a Double-Edged Sword

The "Slow-Thinking" paradigm allows models to reflect and self-verify, but it introduces a significant computational tax. Current research shows two types of redundancy:

1. Repetitive Elaboration: The model describes the same logic in multiple ways despite being confident.
2. Excessive Verification: Under uncertainty (low confidence), the model oscillates between potential paths, often amplifying its own hallucinations through "self-persuasion."

Prior works like ConCISE or TokenSqueeze tried to fix this through global prompts or offline dataset pruning. However, these are "static" solutions to a "dynamic" problem. They don't know when the model is lost and when it is just being wordy.

Methodology: State-Aware Surgery

The core innovation of STACK lies in its ability to diagnose the Reasoning State at every step.

1. Detecting the "Hesitation State"

Instead of just checking for "reflection" keywords (like "Wait," "However"), STACK calculates the Local Information Entropy. If the entropy of the current step $H(X_t)$ significantly exceeds the historical mean, the model is flagged as "hesitating."

2. Knowledge-Guided Contrastive Decoding (KGCD)

When in a hesitation state, STACK doesn't just ask the model to "be brief." It performs Contrastive Decoding:

• Anchor: It retrieves external evidence (e.g., via Bing Web Search).
• Contrast: It penalizes tokens that are favored by the model's inner bias but lack support from external evidence, effectively "resteering" the reasoning toward the truth.

Figure: The STACK framework featuring dynamic switching between self-prompted and knowledge-guided compression.

3. Early Stopping via Answer Convergence

Why keep thinking if the answer is already stable? STACK measures the Information Gain (KL Divergence) between the answer distribution at step $t$ and $t-1$. When the final answer probability distribution stops shifting, the model triggers an "Early Stop," preventing redundant confirmatory loops.

Reward-Difference Driven Training (MDPO)

The authors propose a modified Direct Preference Optimization (MDPO). Instead of a simple binary preference, they use a Reward Margin ($\Delta R$) that accounts for:

• Factuality: Did the compressed version get the right answer?
• Compression Ratio: How many tokens were saved compared to the long-winded original?

Experimental Results: Faster AND Smarter

The results on DeepSeek-R1-Distill-Qwen models are striking. STACK doesn't just make the models faster; it makes them more accurate by eliminating the logical drift that occurs in ultra-long chains.

Figure: Training dynamics showing the adaptive balance between reasoning accuracy and information density.

Critical Insight & Future Outlook

STACK proves that "thinking more" is not always better. By leveraging external knowledge precisely at points of high uncertainty, we can prevent the "hallucination amplification" effect common in long CoT.

Limitations:

• The online retrieval and contrastive sampling add a significant overhead to the training phase (though inference is much faster).
• It currently relies on text-only RAG; extending this to symbolic solvers (like Python interpreters) would likely push mathematical performance even further.

For practitioners, STACK offers a blueprint for building "Self-Correction" modules that are triggered by entropy, rather than fixed schedules—a crucial step toward truly agentic and efficient reasoning engines.