Elite Bases Regression: Breaking the Speed Barrier in Symbolic Regression

TL;DR

Elite Bases Regression (EBR) is a breakthrough non-evolutionary algorithm that transforms symbolic regression from a slow, stochastic search into a deterministic, real-time process. By using a "selection of the fittest" approach based on correlation coefficients rather than random mutations, it produces more accurate and concise mathematical models than current SOTA methods like FFX, effectively solving the bottleneck of computational complexity in large-scale data mining.

Background: Beyond the Evolutionary Plateau

Symbolic regression (SR) is the "holy grail" of data mining: it doesn't just predict values; it finds the underlying mathematical laws. For decades, Genetic Programming (GP) has dominated this field. However, GP is notoriously slow and inconsistent. If you run it twice, you might get two different answers.

The transition to non-evolutionary, deterministic methods like Fast Function eXtraction (FFX) marked a shift toward speed, but FFX hits a wall when the number of basis functions grows, as it relies on solving complex quadratic optimization problems. EBR enters the scene by asking: What if we only keep the best bases from the start?

Methodology: How EBR Works

The core of EBR lies in two innovative components: Parse-Matrix Encoding and the Elite Preservation Strategy.

1. Parse-Matrix Encoding

Instead of complex tree structures used in GP, EBR uses a compact matrix representation. Each row in the matrix corresponds to a specific operation (like sin, exp, or +) and its associated variables. This allows the algorithm to systematically "enumerate" possible functions without getting lost in an infinite search space.

2. The Elite Preservation Strategy

While FFX tries to calculate weights for a massive set of functions simultaneously, EBR acts as a filter.

• Correlation Filtering: It evaluates every generated base against the target data using correlation coefficients ($\rho$).
• Basis Pruning: If a base doesn't "look" like the target landscape, it is immediately discarded.
• GLM Spanning: Only the top-performing "elite" bases are fed into the Generalized Linear Model (GLM) for the final coefficient optimization.

Performance: EBR vs. FFX

The researchers tested EBR against FFX across 28 challenging cases, ranging from simple polynomials to highly nonlinear trigonometric functions.

Key Findings:

• Exact Recovery: EBR successfully found the exact hidden formula for 16 out of 16 "exact fitting" problems, whereas FFX often produced bloated approximations.
• Error Reduction: In Case 11 ($\sin (x_1^2-x_2)$), FFX failed significantly with a 28% error, while EBR achieved near-perfect recovery.
• Model Conciseness: EBR models are typically much simpler and more "human-readable," using fewer terms to achieve higher accuracy.

Deep Insight: Why Correlation Matters

The "magic" of EBR is that even if a single basis function (like $x_1{x}_2$) isn't the perfect answer, its correlation coefficient will reveal if it captures the landscape of the target function. By treating functions as vectors in high-dimensional space, EBR can "sketch out" the solution before fine-tuning the details. This is why it can handle highly nonlinear behavior that confuses standard machine learning models.

Conclusion and Future Outlook

EBR represents a significant step towards Real-time Symbolic Regression. By replacing the "random walk" of evolution with a structured, correlation-based selection, it offers:

1. Repeatability: Deterministic results every time.
2. Efficiency: Minimal memory overhead and execution time.
3. Accuracy: Superior handling of complex nonlinearities.

Future work aims to introduce nonlinear correlation detection, which could allow EBR to tackle even more chaotic real-world industrial data, such as aerodynamic coefficient prediction or complex signal processing.

Takeaway for Practitioners

If your task requires finding a clear, explainable mathematical model from data in a short timeframe, EBR's elite preservation approach is a superior alternative to traditional GP or regularization-heavy methods.