LGF: Bridging the Gap Between Neural Flows and Symbolic Physics

TL;DR

Discovering the governing laws of a dynamical system is often a "needle in a haystack" problem. Latent Grammar Flow (LGF) turns this into a guided search. By embedding mathematical grammars into a discrete latent space and using a flow-based model to navigate it, LGF discovers ODEs that aren't just accurate, but physically "sane" (stable and interpretable).

The "Structure vs. Behavior" Paradox

In symbolic regression, two equations might look nearly identical (e.g., $u^{\prime }=\sin (t)$ vs. $u^{\prime }=\sin (10t)$) but behave radically differently. Standard AI models often fail because they focus on the syntax (the tokens) rather than the semantics (the trajectory).

The authors identify a critical flaw in prior work: continuous latent spaces (like VAEs) often produce "illegal" equations when decoded, while discrete searches (like Genetic Programming) are blind to the underlying physics until the very end.

Methodology: GQAE and Behavioral Flows

LGF introduces a two-pronged innovation to solve this:

1. Grammar Quantization Autoencoder (GQAE): Instead of mapping equations to a messy continuous cloud, LGF uses Finite Scalar Quantization (FSQ) to map grammar rules to a clean, discrete grid. This respects the "all-or-nothing" nature of mathematical symbols.
2. Behavioral Latent Distance: This is the secret sauce. During training, the model doesn't just learn to reconstruct the text of the equation. It calculates the Wasserstein distance between the vector fields produced by different equations.
   • Insight: If two equations produce similar movement in phase space, they are forced to be close in the latent space.

Figure 1: The GQAE Pipeline. Note the Semantic Loss path that uses the Wasserstein distance to align the latent space with physical behavior.

Guiding the Flow

Once the latent space is organized, the model uses Discrete Flow Matching. Unlike standard Diffusion, this model "flows" from random noise towards high-probability equation candidates. More importantly, it uses Predictors to steer the flow toward equations that are:

• Stable: No more exploding solutions that violate physics.
• Low Complexity: Preferring $F=ma$ over a 10th-order polynomial mess.

Breakthrough Results

LGF was tested against heavyweights like PySR (the industry standard for symbolic regression) and ODEFormer (a Transformer-based approach).

In Benchmark 3 (Real-world structural dynamics including the Van der Pol oscillator), the difference was stark:

• PySR often found "trick" equations that had low residual error but completely failed to follow the actual path of the system when solved.
• LGF consistently recovered equations where the numerical solutions matched the ground truth, even with 5% noise in the data.

Figure 2: Performance on Benchmark 2 showing LGF's ability to mirror ground truth trajectories across displacement, velocity, and acceleration.

Critical Analysis & Conclusion

LGF is a significant step toward "AI Scientists." It treats physics not as a post-processing check, but as a core part of the generative process.

Limitations: Currently, the framework is validated on one-dimensional ODEs. While the authors suggest it can scale to higher dimensions, the complexity of the grammar and the computational cost of the stability check (Lyapunov's method) might become bottlenecks for chaotic N-body systems.

Final Takeaway: The success of LGF suggests that the future of scientific AI isn't just "bigger transformers," but rather smarter inductive biases—teaching models to respect gravity, stability, and the rules of the logic before they ever see a single data point.