Perturbation is All You Need: Scaling LLM Extrapolation via Semantic Noise

TL;DR

Modern LLMs struggle with "unseen" prompts because they are trained to be hypersensitive to exact token sequences. This paper proposes a paradigm shift: Perturbed Autoregressive Modeling. By injecting semantic noise (like synonym swaps or insertions) during both training and inference, we can force models to map out-of-distribution inputs back to known semantic manifolds. The result? A model that doesn't just memorize, but extrapolates.

Background: The Extrapolation Wall

As we approach the limits of high-quality human text data, the "Scaling Laws" are hitting a plateau. The core issue is Extrapolation Uncertainty. Standard models are essentially "Post-additive noise models"—the noise happens only at the output (sampling). When these models face a prompt that lies far from their training support, their performance collapses because they lack a mechanism to bridge the gap.

The authors argue that human cognition stays robust by using metaphors and analogies—we recognize that "full of love" and "a lot of love to give" are semantically identical despite different syntax.

Methodology: Shifting to Pre-Post-Additive Noise

The breakthrough here is the transition from a simple autoregressive chain to a hierarchical model.

1. Perturbation (Step 1): Take the prefix $X_{<t}$ and transform it into $ilde{X}_{<t}$ using a perturber $T$.
2. Inference (Step 2): Predict the next token $X_t$ based on the perturbed prefix.

The Architecture Shift

Unlike traditional Data Augmentation, which is discarded after training, this method treats $T$ as part of the model's DNA.

Figure 1: Comparison between standard LLM training and the Perturbed Autoregressive framework.

For training efficiency, the authors maximize a lower bound of the log-likelihood: ${\mathcal{L}}^{Log}(P_{h}eta;X):=\frac{1}{m}{\sum }_{i=1}^m{\sum }_{t=1}^L\log P_{h}eta(X_t|ilde{X}_{<t}^{(i)})$ This ensures the base model $P_{h}eta$ becomes invariant to the specific "noise" injected by $T$.

Theoretical Guarantees: Adaptivity & Robustness

The paper provides a rigorous mathematical framework defining Extrapolability. They prove three vital properties:

• Adaptivity: If your base model can extrapolate, the perturbed version inherits that power.
• Robustness: Even if the base model is poor at extrapolating, the perturber $T$ provides a "safety floor" for performance.
• Perfect Extrapolation: If the perturber can map any unseen prompt back to a "neighbor" in the training set, the uncertainty drops to zero.

Experiments: Breaking the SOTA

The authors tested their framework on various architectures (GPT-2, OPT, GPT-Neo) across diverse datasets.

Performance Gains

In out-of-sample tasks like WritingPrompts, which require high creativity and reasoning, the Perturbed MLE consistently crushed standard MLE and even advanced techniques like EMO (Earth Mover's Distance Optimization).

Table 1: Mauve and ROUGE-1 scores across multiple models and datasets.

The Necessity of Dual-Stage Perturbation

One of the most insightful parts of the study is the Ablation Study. If you only perturb during training (Data Augmentation) or only during testing, the gains vanish. The model must learn the "perturbed manifold" during training to successfully navigate it during inference.

Figure 2: Mauve score differences showing that only 'Both' (Train + Test) perturbation provides substantial delta.

Critical Insight & Conclusion

This work suggests that the "brute force" scaling of data might not be the only way forward. By mathematically modeling the "semantic neighborhood" of language, we can build models that are fundamentally more stable.

Limitations: The current perturbers (random insertion/deletion) are quite "coarse." A future direction would be using a parameterized, learnable perturber (perhaps another small LLM) that can be jointly optimized with the base model to find the most "informative" semantic neighbors.

Takeaway: If we want LLMs to truly "reason" and "create" rather than "retrieve" and "parrot," we must stop training them on exact strings and start training them on the underlying semantic latent space.