Sample More, Reflect Less: Why Your LLM's "Self-Reflection" Might Be a Waste of Tokens
Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B
The paper investigates whether popular LLM reasoning methods (e.g., Self-Refine, Reflexion, Multi-Agent Debate) actually improve performance when controlled for "token budget." It introduces a robust sampling baseline, demonstrating that simply sampling more attempts and taking a majority vote (Self-Consistency) consistently outperforms or matches complex iterative methods at equal token costs.
TL;DR
In the quest to make Large Language Models (LLMs) smarter, we often tell them to "think again," "critique your answer," or "debate with yourself." This paper reveals a sobering reality: these complex reasoning strategies almost always lose to the simplest possible baseline—repeated sampling with majority voting—when you hold the token budget constant. If you have 2,000 tokens to spend on a math problem, you are better off generating five independent solutions than one solution with three rounds of "self-reflection."
The "Budget" Illusion
The academic landscape is currently flooded with "reasoning" wrappers like Self-Refine, Reflexion, and Multi-Agent Debate. These methods frequently report 5-10% accuracy gains over a single Chain-of-Thought (CoT).
However, the author, Iliya Mirzaei, points out a glaring logical flaw: Iterative methods are expensive. A single CoT might use 300 tokens, while a debate uses 3,000. If the debate performs better, is it because the "debate" format is superior, or simply because the model had 10x more "thinking" space?
Methodology: The Equal-Cost Battleground
To settle this, the researchers compared seven popular methods across three model sizes (Qwen2.5-1.5B, 3B, and 7B) on two math benchmarks (GSM8K and MATH-500).
The core of their approach was the Sampling Baseline Curve. Instead of comparing a method to a single CoT, they generated a pool of 16 samples for every question. This allowed them to calculate exactly how accurate "Majority Voting" (Self-Consistency) would be at any token cost.
Figure 1: The gap between various reasoning methods and the Sampling Baseline. Note how almost all iterative methods (Self-Refine, Reflexion) fall below the 0-line, meaning they perform worse than simple sampling at equal cost.
The Verdict: Judging is Harder than Solving
The most striking finding involves Best-of-N. In this method, the model generates 8 answers and is then asked to "pick the best one."
The researchers took those exact same 8 answers and compared two strategies:
- Judging: Ask the model to pick the best.
- Counting: Take the most common answer (Majority Vote).
Counting won every single time. On the 3B model, counting was a staggering 17.3 percentage points more accurate than judging.
Table 1: The "Judging vs. Counting" showdown. Even with the same candidates, the model's choice is significantly less reliable than a simple tally.
Why Does This Happen?
The paper suggests two critical insights:
- The "Correctness" Recognition Gap: A model might be smart enough to produce a correct answer once in eight tries, but it is often not smart enough to recognize that answer as correct among seven wrong ones.
- Silent Degeneration: The researchers found that Reflexion, as originally described, often "faked" its success. On small models, the model would look at its first answer, declare it "CORRECT" (even if it was wrong), and stop immediately. It scored well only because it accidentally became a cheap, single-pass CoT.
Does Scale Save the Day?
As models grow to 7B parameters, the performance gap between "counting" and "judging" begins to close. However, the author notes that this isn't because the model becomes a brilliant critic. Rather, the model simply begins to agree with the majority more often. The "untrained" verifier inside the LLM eventually reaches parity with a simple vote, but it rarely surpasses it.
Takeaways for AI Engineers
- Don't over-engineer prompts: Before implementing a complex "Reflection" loop, try running 5-10 samples with a majority vote. It's easier to implement and likely more accurate.
- Measure Token Cost: Never trust a paper that claims a "reasoning" improvement without a cost-matched baseline.
- The Power of Tallying: Probability is a more robust tool than LLM self-judgment for objective tasks like math and logic.
Conclusion
This study serves as a "Occam's Razor" for LLM inference. While the "Reflection" and "Agentic" hype is exciting, the most reliable way to get a better answer from a model is simply to let it try several times and see where the consensus lies. Sample more, reflect less.