[ICML 2025] GeoPT: Breaking the Physics Data Bottleneck via Lifted Geometric Pre-Training

TL;DR

Training high-fidelity neural simulators normally requires thousands of hours of expensive CFD (Computational Fluid Dynamics) data. GeoPT changes the game by pre-training on 1 million samples using only raw 3D geometries (like those from ShapeNet) and synthetic "fake" dynamics. By teaching the model how particles would walk across a surface, it creates a physics-aware prior that cuts down the need for real simulation data by up to 60%.

The "Geometry-Physics Gap"

In the world of AI for Science, we have a data asymmetry problem. We have millions of 3D CAD models (geometries), but we only have a handful of high-fidelity simulations for them.

Standard self-supervised learning (like Masked Autoencoders) fails here. Why? Because a static shape doesn't tell you how wind or force flows around it. As the authors visualised (see below), models pre-trained only on static geometry learn symmetries that are physically nonsensical in a wind tunnel. This is the Geometry-Physics Gap.

Fig 1: Notice how geometry-only pre-training (left) fails to capture the front-back asymmetry essential for aerodynamics, while physics-aware training (right) captures the flow structure.

Methodology: The Power of "Lifting"

The core innovation of GeoPT is Dynamics-Lifted Geometric Pre-Training.

Instead of just looking at a point $x$ on a car, GeoPT:

1. Assigns a random velocity vector $v$ to that point.
2. Calculates a "synthetic trajectory" (where would that point go if it followed $v$ until it hit a boundary?).
3. Asks the model to predict this trajectory.

This "lifts" the model’s understanding from 3D space ($x,y,z$) to a higher-dimensional phase space $(x,v)$. This essentially teaches the model mass conservation and boundary interactions—the bread and butter of all physics—without ever running a single "real" simulation during pre-training.

Fig 2: GeoPT uses random dynamics to "prompt" the model during pre-training, which is then replaced by real task-specific velocities (e.g., 100mph wind) during fine-tuning.

Experimental Results: Scaling the Simulation

The authors tested GeoPT on five industrial-scale benchmarks:

• DrivAerML: External car aerodynamics.
• NASA-CRM: Aircraft pressure fields.
• DTCHull: Ship resistance in water.
• Car-Crash: Structural stress during collisions.

Key Performance Highlights:

• Data Efficiency: Reached the same accuracy as a standard model using 40% less data.
• Convergence: Trained 2x faster.
• Architecture Agnostic: While based on the Transolver backbone, the method works for various Transformer-based operators.

Fig 3: GeoPT (Red) consistently outperforms "Training from Scratch" (Blue) across different data scales and epochs.

Deep Insight: Why Does It Work?

The magic lies in Remark 4.1 of the paper: The pre-training task is mathematically equivalent to solving a transport equation ${\partial }_{t}f+v\cdot abl{a}_{x}f=0$. By forcing the neural network to satisfy this conservation law across millions of random shapes, the model develops an inductive bias for how physical fields propagate.

When you finally show it real CFD data, it doesn't have to learn "what is a boundary" or "how does air move"—it just needs to learn the specific "flavor" of the Navier-Stokes equations for that task.

Future Outlook & Limitations

While GeoPT is a massive leap forward, it still relies on a point-wise velocity parameterization. It doesn't yet account for complex material properties (like elasticity or varying thickness) in its pre-training "prompt."

However, its ability to generalize to Radiosity (light transport) suggests that GeoPT might be the precursor to a true Foundation Model for Engineering, capable of switching between fluids, solids, and electromagnetics simply by changing its "velocity prompt."

Conclusion

GeoPT proves that we don't need million-dollar solver runs to build smart simulators. We just need to teach AI the "common sense" of motion using the geometries we already have.

Main Takeaway: By augmenting static 3D data with synthetic trajectories, GeoPT bridges the geometry-physics gap, unlocking industrial-fidelity neural simulation at a fraction of the data cost.