Compiling Lean code is not semantic preservation
Autoformalisation here means an AI system translating a mathematical text from natural language into a formal language such as Lean, after which the formal argument can be mechanically checked. The paper separates two tasks that are easy to conflate. The first asks only that the AI produce Lean code which compiles, without the `sorry` keyword or extra axioms, for an alleged natural-language proof. The second asks for a semantically faithful translation of every definition, statement and proof, preserving the source's mathematical content and the dependencies between results.
The paper's central warning is that success at the first task says nothing about the second. Lean acceptance of a translation does not imply semantic preservation, because many candidate translations would compile, some faithful and many not. Lean therefore provides no safety net if the AI chooses a mistranslation, and the correctness of the formal proof says nothing about the correctness of the natural-language proof.
The authors are explicit about the scope of this claim: they do not assert that OpenAI's natural-language Navier-Stokes proof is wrong. They make statements about mistranslations into Lean, and about what a Lean verification can and cannot certify.
Resolving ambiguity sits at unbounded SCI
To translate faithfully, an AI must resolve ambiguities in mathematical natural-language text, including whether a defined object actually exists. The paper formalises this as a presupposition-resolution problem and places it in the Solvability Complexity Index hierarchy, which generalises the arithmetical hierarchy. The result is that the ambiguity-resolution problem has SCI equal to infinity.
The comparison the authors draw is stark: the Halting problem has SCI equal to 1, so semantically faithful autoformalisation is informally harder than any computational problem, meaning any problem with finite SCI. The paper states this precisely as a requirement on a trustworthy autoformaliser: one that always outputs either a semantically faithful translation or a declaration that the statement cannot be translated must solve problems harder than any problem with finite SCI.
The proof builds this from least-number constructions whose well-definedness depends on whether a certain set of natural numbers is empty. Corollary A.5 shows that for each level l at least 2, the corresponding decision problem has SCI exactly l, so the problem of answering the question for all levels has SCI equal to infinity. This is a result about the difficulty of the translation task, not a measurement of how often current models fail.
A wrong natural-language proof can compile in Lean
The paper's simplest demonstration uses a polynomial inequality. The natural-language proof claims that p(x) = x^3 - x^2 - x + 1 has roots -1 with multiplicity two and 1 with multiplicity one, and concludes p(x) is nonnegative for x at least -1. The stated multiplicities are wrong, so the claimed factorisation is wrong too.
When a user asked a chatbot to write that argument as a compiling Lean proof, the chatbot returned a correct Lean proof by finding the correct multiplicities. The formal proof is valid, but it is not a translation of the supplied argument. This is the clearest case of the mismatch: the natural-language proof is incorrect, the Lean proof is correct, and the compilation result tells the reader nothing about the original argument.
A second example shows the same gap without an error. For a real symmetric matrix A, the natural-language proof diagonalises A in an orthonormal eigenbasis and reduces the trace of A squared to a sum of squared eigenvalues. The Lean proof instead keeps the original matrix and uses symmetry to obtain a sum of squares of the entries. Both proofs are correct, yet the Lean argument is distinctly different from the one the user asked to have formalised.
OpenAI's Lean estimate needs one more derivative
The paper's most concrete technical comparison concerns Lemma 8.6, equation (8.19), of OpenAI's announced Navier-Stokes blow-up proof. In the natural-language paper, the inverse estimate bounds the C^m norm of N^{-1}F by a constant times the C^{m+4} norm of F. In the cited Lean declaration, the corresponding bound requires the C^{m+5} norm of F.
The extra derivative is not a cosmetic difference. The natural-language proof uses the convergent series with terms of order (1+|k|)^{-3} in two dimensions, which is what four additional derivatives buy. The Lean code instead invokes a coefficient seminorm bound that produces a series with terms of order (1+|k1|+|k2|)^{-4}, requiring five additional derivatives. The paper notes that similar m+5 requirements appear in other Lean declarations, including `inverse_finiteJets` and `mixedJet_inverse_bound`.
Because the Lean estimate is strictly weaker than the natural-language estimate, the formalisation does not establish the statement the natural-language paper states. The paper also notes that some Lean results are incomparable with the natural-language bound because the derivative types differ, for example where Lean theorems use mixed derivatives rather than only torus derivatives.
The pressure-flux bound changes statement and proof
A second comparison concerns equation (10.19) of the natural-language paper, a pressure-flux estimate used inside the proof of a velocity uniqueness lemma. The natural-language bound is stated purely in terms of B_R, a localised L^6 velocity norm. The Lean estimate additionally depends on A_R, a localised L^2 gradient norm, and the two bounds are therefore hard to compare.
The paper traces the divergence to three changes in proof method. The Lean proof does not rely on boundedness of the Riesz transform as a map from L^{3/2} to L^{3/2}; it instead invokes the Sobolev embedding to move to L^2 and uses Riesz boundedness there, and the extra derivative from that embedding is why A_R appears. The Lean proof also applies Hölder's inequality with different exponents, changing the final bounds, and it explicitly shows the left-hand integral is continuous in time, upgrading an almost-everywhere statement to one without that qualifier.
The paper also records a structural difference in how the integral is split: the natural-language proof factors the integrand using a test function containing φ_R^3, while the Lean proof splits it using φ_R^2 and a test function containing φ_R^5. The authors note that the natural-language bound is in fact stricter, since the purpose of the lemma is to establish that w vanishes, but that this is only established after the entire proof of Lemma 10.5 is complete.
What a Lean check can and cannot certify
The paper's practical conclusion follows from the distinction between the two autoformalisation tasks. If the goal is to know whether a natural-language argument is correct, the relevant task is faithful semantic translation, and that task is the one shown to have unbounded SCI. A Lean compilation of an AI translation addresses the easier task, where the AI is free to produce any compiling proof of the stated theorem.
The authors therefore argue that natural-language proofs by OpenAI and other autoformalised Lean proofs should not be trusted prima facie without the same peer review and scrutiny applied to other proofs. The Lean code tells us that the theorem is correct, but the natural-language proof with its intermediate arguments, lemmas and results may be incorrect or may contain incorrect arguments.
The paper does not claim to show how to build a trustworthy autoformaliser; it states that this is beyond its scope and is a major open problem. It does claim that even the most advanced OpenAI models are far from achieving semantic preservation. The specific evidence for that claim is the set of mistranslations documented here, not a systematic benchmark of model accuracy.
Where the mismatch matters most
The comparisons in this paper are not all of one kind, and the difference matters for how much weight each carries. The polynomial and matrix examples show that a compiling Lean proof can be unrelated to the supplied natural-language argument, including when that argument is wrong. The Navier-Stokes comparisons go further: in the derivative estimate, the Lean statement is strictly weaker than the natural-language statement, and in the pressure-flux case both the bound and the mathematical argument used to establish it differ.
That ordering is the paper's most useful takeaway for readers assessing autoformalised mathematics. A Lean check is strongest as evidence about the formal statement it actually encodes, and weakest as evidence about the natural-language proof that was supposed to be translated. The gap widens as the formal statement departs from the natural-language one, whether by an extra derivative, an added dependence on A_R, or a different proof strategy.
The paper's own priority is therefore the faithful-translation task, not the compiling-proof task. Its unbounded-SCI result explains why that priority cannot be discharged by better engineering alone, and its Navier-Stokes examples show what the failure looks like in a high-profile case: the formalisation can be valid while the natural-language argument it was meant to certify remains unverified.
Sources used in this answer
Navier-Stokes lost in translation: Why Lean verification of AI autoformalisation does not guarantee correct natural language proofs
Lean acceptance of an AI-translated proof does not imply semantic preservation; resolving ambiguities for faithful translation has SCI equal to infinity, and documented mistranslations include OpenAI's Navier-Stokes formalisation, where the Lean derivative estimate requires m+5 derivatives instead of the natural-language m+4 and the pressure-flux bound depends additionally on A_R.