Lattice Deduction Transformers: Solving the "Unsolvable" Sudoku with 800K Parameters

TL;DR

While frontier LLMs (GPT-4/5, Claude 3.5/4) with trillions of parameters frequently fail at basic logical puzzles, the Lattice Deduction Transformer (LDT)—a tiny 800K-parameter model—achieves 100% accuracy on Sudoku-Extreme. It achieves this by abandoning the "Chain-of-Thought" generation paradigm in favor of Abstract Interpretation: it projects its internal thoughts onto a mathematical lattice, ensuring every step is a logically sound deduction rather than a lucky guess.

Background: Why LLMs Fail at Logic

Reasoning in modern AI is dominated by Chain-of-Thought (CoT). However, CoT is essentially "token-level dreaming"—it lacks a formal grounding mechanism to verify if an intermediate step is logically possible. In contrast, classical solvers use Constraint Satisfaction to prune impossible branches. LDT bridges this gap by embedding a symbolic lattice directly into the Transformer's recurrent loop.

Methodology: Thinking in Lattices

The core innovation is the Lattice Projection. Instead of passing hidden vectors that only the model understands, LDT maps its latent state to a Grid Powerset Lattice.

1. Lattice Encoding

For a Sudoku grid, the model tracks which digits are still "viable" for each cell using multi-hot encodings. A deduction step is defined as pushing the confidence sigmoids of impossible candidates toward zero.

Figure 1: The LDT recurrent loop, alternating between neural deduction and stochastic branching.

2. The Alpha ($\alpha$) Operator

To train the model, the authors use the $\alpha$ operator from Abstract Interpretation. This operator takes the set of all possible valid solutions and collapses them into the most precise "abstract" state. This allows the model to receive on-policy supervision: it is trained to match the best possible deductive move from its current (potentially messy) state.

3. Soundness vs. Completeness

The model is designed to be empirically sound. It has two heads:

• Candidate Head: Eliminates impossible digits.
• Conflict Head (CLS): Flags when a branch has reached a dead end (inconsistency). If the model isn't sure, it triggers backtracking rather than hallucinating a wrong digit.

Experiments & Results: David vs. Goliath

The performance gap between LDT and frontier LLMs is staggering. On Sudoku-Extreme (puzzles requiring deep search), Claude 4.6 and GPT-5.4 score 0%, whereas LDT hits 100%.

Figure 2: The train/test compute trade-off. As the model trains more, the number of forward passes (inference cost) drops drastically as the model gets "smarter" at deduction.

Key Metrics:

• Sudoku-Extreme: 100% Solve Rate (vs. 0% for LLMs).
• Generalization: A single LDT model solved Snowflake Sudoku (hexagonal grids) of varying sizes, proving it isn't just memorizing 9x9 grids.
• Maze-Hard: 99.9% accuracy on 30x30 mazes by learning to intersect multiple shortest paths using the $\alpha$ operator.

Deep Insight: Training Shortens Search

One of the most profound takeaways is the Inference Scaling behavior. The paper demonstrates that more training (Pre-computation) directly translates to shorter search paths at test time. The model essentially "learns" the symbolic shortcuts of the puzzle, moving from a brute-force search mode to a direct "deduction mode."

Critical Analysis & Future Work

Limitations: LDT currently excels where rules are fixed (Sudoku, Mazes). However, on the ARC-AGI benchmark—where rules change every task—the model plateaus. The conflict head becomes unreliable when the model must "infer the rules" before applying them.

Future Outlook: LDT suggests a future where we don't need trillion-parameter models for logic. Instead, we can use "Neural Solvers" that treat reasoning as a fixed-point computation on a lattice. This could revolutionize hardware verification, software synthesis, and any field where "almost correct" is not good enough.

Author's Note: This work proves that "Small Data + Correct Inductive Bias" beats "Big Data + Opaque Architecture" for hard reasoning.