Neural Thickets: Why Random Guessing is a Valid Post-Training Strategy for LLMs

TL;DR

"Guess and check" is usually the first algorithm we learn, and the last one we'd apply to a billion-parameter LLM. However, Neural Thickets reveals that for large, well-pretrained models, the space surrounding the weights is "thick" with diverse task experts. By simply sampling random perturbations and ensembling them (RandOpt), we can match the performance of complex RL algorithms like PPO in a fraction of the wall-clock time.

Problem & Motivation: The Needle vs. The Thicket

In the early days of AI, random search was dismissed because the probability of "stumbling" upon a solution in high-dimensional space was considered astronomically low—the "needle in a haystack" regime.

The authors of this MIT CSAIL study argue that pretraining changes everything. As models scale, they transition from a regime where solutions are rare to a Thicket Regime, where the neighborhood of pretrained weights is replete with specialized solutions. The core insight is that the "foundation model" isn't a single point, but a seed for a distribution of potential experts.

Methodology: High-Density, High-Diversity

The paper defines two critical metrics to characterize this landscape:

1. Solution Density: The probability that a random perturbation improves task performance.
2. Spectral Discordance: A measure of how "specialized" these perturbations are (whether an improvement in Math hurts Chemistry).

The RandOpt Algorithm

Rather than performing sequential gradient steps, RandOpt takes a "massively parallel" approach:

• Phase 1 (Training): Perturb the base weights $N$ times using Gaussian noise $ϵ\sim \mathcal{N}(0,{\sigma }^{2}I)$.
• Phase 2 (Selection): Rank these $N$ "guesses" on a small training set and keep the top $K$.
• Phase 3 (Inference): Use a majority vote across the $K$ experts to generate a final answer.

Figure: Topographical shift—as models scale (left to right), the "blue" areas of degradation are replaced by "red" basins of improvement.

Experiments & Results: Sampling vs. Optimization

The authors tested RandOpt against heavyweights like PPO (Proximal Policy Optimization) and GRPO.

Key Findings:

• Scalability: The "Thicket" only emerges at scale. Small models (like GPT-2 0.1B) gain nothing from RandOpt. However, for models >1.5B, performance spikes.
• Diversity Wins: The perturbations are specialists. One might be great at Python, another at Creative Writing. Aggregating them via majority vote is essential to surpass the base model.
• Wall-Clock Efficiency: Because RandOpt is $O(1)$ in sequential steps, it can be finished in minutes on a large GPU cluster, compared to hours for iterative RL methods.

Figure: RandOpt (stars) vs. various baselines. Note how RandOpt consistently hits the SOTA frontier across different reasoning tasks.

Critical Analysis: Is it just "Formatting"?

A valid criticism of any post-training method is whether it's actually teaching "reasoning" or just fixing "format" (e.g., putting an answer in <answer> tags).

The authors performed an ablation study on GSM8K and found that while a significant portion of the gain comes from Format Thickets (fixing how the answer is presented), there is a genuine Reasoning Thicket—a 12.3% boost where the model actually solves problems it couldn't solve before.

Conclusion & Future Work

Neural Thickets suggests that "Pretraining is all you need." The heavy lifting of learning representations is done early; post-training is merely a matter of selecting from the nearby solutions the model already intuitively "knows."

Limitations:

• Inference Cost: The $K$ forward passes make inference expensive, though the authors demonstrated that these $K$ experts can be distilled back into a single fast model with only a 2% increase in training cost.
• New Skills: It is unlikely random guessing can teach a model a completely new language; it essentially "sharpens" latent skills.

This work reframes our understanding of LLM "alignment"—it's not a search for a new person, but a selection from a crowd of experts living inside the same set of weights.