Do LLMs Benefit from Their Own Words? Unveiling the "Context Pollution" Trap
Do LLMs Benefit From Their Own Words?
This paper investigates the necessity of retaining prior assistant responses in LLM conversation histories. By comparing "Full Context" (FC) with "Assistant-Omitted" (AO) prompting across four models (Qwen3-4B, DeepSeek-R1-8B, GPT-OSS-20B, and GPT-5.2), the researchers demonstrate that omitting assistant-side history often maintains response quality while reducing context usage by up to 10x.
In the world of Large Language Models (LLMs), the prevailing mantra has always been: Context is King. We carefully preserve every turn of a conversation, assuming that more history leads to better understanding. But a provocative new study from MIT and IBM Research suggests we might be poisoning our models with their own generated words.
TL;DR
The researchers found that for a significant portion of real-world conversations, omitting the assistant's previous responses does not hurt performance—and can actually prevent errors. By shifting to a "User-Turn-Only" context, researchers achieved up to a 10x reduction in context length while maintaining, or in some cases improving, response quality.
The "Context Pollution" Problem
We usually blame the user for messy prompts, but this paper identifies a phenomenon called Context Pollution. Because LLMs (especially reasoning models) generate long, speculative, and sometimes error-prone "thinking traces," they often get distracted by their own previous hallucinations or stylistic artifacts.
For example, if a model mistakenly introduces a specific parameter in Turn 1, it often forces that same parameter into Turn 5, even if it's no longer applicable, leading to bugs or logic failures.
Methodology: The Assistant-Omission Experiment
The authors compared two primary configurations:
- Full Context (FC): The standard approach (All User + All Assistant turns).
- Assistant-Omitted (AO): Only prior User turns are kept; Assistant turns are replaced with a simple placeholder:
[Response provided].
To understand why AO works, they classified prompts into three categories:
- New Ask (36.4%): Standalone queries mid-conversation (e.g., "How do I call a function in Python?").
- Follow-up with Feedback (30.5%): Direct instructions to change a previous output.
- Follow-up without Feedback (33.1%): References to prior turns without clear instructions.
Figure 1: Comparison of the model carrying over a bug (Jaccard metric) in Full Context vs. generating clean code in Assistant-Omitted context.
Key Findings: Less is More
The results were surprising across the board, affecting even frontier models like GPT-5.2:
- The 10x Compression: By round 8, standard histories grew to 55,000 characters, while AO context stayed near 10,000.
- Open Models Thrive on Silence: Reasoning models like DeepSeek-R1-8B and GPT-OSS-20B maintained quality even when their own words were purged.
- The Pollution Examples: The authors documented cases in math, coding, and literature where models "anchored" to their own previous mistakes.
Figure 3: Win rates by prompt category show that "New Asks" perform identically without assistant history.
Adaptive Assistant-Response Omission
Recognizing that some prompts do need prior context (e.g., "What did you mean by that?"), the authors built an Adaptive Context Manager. They trained a simple classifier to predict if a turn really needs the assistant's history.
Figure 5: The "Efficient Frontier" of the adaptive strategy, maintaining 95% performance with significantly fewer tokens.
Technical Insight: Why Does This Happen?
The physics of this result lies in Inductive Bias and Over-conditioning. Once a model sees its own generated output in the KV cache, it treats those tokens as "ground truth" for the style and factual boundaries of the conversation. If that output is a "thinking trace" (like in DeepSeek-R1), it contains high-entropy, speculative noise. By removing it, we force the model to re-ground its reasoning solely in the user's requirements.
Critical Analysis & Future Outlook
While this is a breakthrough for inference efficiency, it suggests that our current benchmarks for "Long Context" may be flawed. If a model can answer a question without 80% of the history, is it really "reasoning" over long context, or just ignoring the noise?
Takeaway: For developers building agents or chat interfaces, the default should no longer be "send everything." Implementing a simple "Omit Assistant History on New Ask" heuristic could immediately yield a 30-50% reduction in KV cache costs with zero loss in quality.
The future of context management isn't just about fitting more tokens—it's about knowing which ones to forget.