[ICLR 2026] SE-RRM: Scaling Reasoning via Architectural Symbol-Equivariance

TL;DR

Current AI models struggle with abstract reasoning because they treat every symbol (like the number '1' in Sudoku) as a unique feature to be learned. SE-RRM (Symbol-Equivariant Recurrent Reasoning Model) changes this by architecturally enforcing that rules are independent of specific symbols. With just 2 million parameters, it not only crushes SOTA benchmarks on Sudoku but also generalizes to larger puzzles (16x16, 25x25) that prior models literally could not process.

The Blind Spot of Current Reasoning Models

Why do Large Language Models (LLMs) and even specialized Recurrent Reasoning Models (RRMs) like HRM or TRM require massive data augmentation to solve a simple puzzle?

The answer lies in Symmetry. In a Sudoku puzzle, the rules remain identical whether you use numbers 1-9, letters A-I, or nine different emoji. However, standard neural networks assign a different "embedding" to each symbol. To the model, '1' and '2' are as different as 'cat' and 'dog'. To learn that they are interchangeable participants in a rule-set, we usually have to show the model thousands of permuted versions of the same puzzle.

Existing RRMs hit a wall when:

1. The number of symbols changes: A model trained on 9x9 Sudoku (9 digits) cannot even "see" a 16x16 Sudoku (16 digits) because it lacks embeddings for digits 10-16.
2. Data is scarce: In benchmarks like ARC-AGI, we don't have millions of examples to teach the model every possible color permutation.

The Solution: Symbol-Equivariance

The authors of SE-RRM introduce an architectural constraint: the model's output must change in the exact same way the input symbols are permuted. If you swap "Red" and "Blue" in the input, the output must swap "Red" and "Blue" exactly.

How it Works: The 3D Reasoning Tensor

Instead of a flat 2D matrix (Positions × Features), SE-RRM uses a 3D tensor:

• Dimension 1: Features ($D$)
• Dimension 2: Positions ($I$)
• Dimension 3: Symbols ($K$)

The logic is split into two specialized attention operations:

1. Position Attention ($T^{D,I}$): Reasons about where things are (e.g., "Is there a conflict in this row?").
2. Symbol Attention ($T^{D,K}$): Reasons about what things are (e.g., "Which symbols are still available for this cell?").

By sharing the same embedding vector for all "usual" symbols and only differentiating them via their position in the $K$-dimension, the model learns the logic of the grid rather than the identity of the digits.

Experimental Results: True Extrapolation

The most striking result is SE-RRM's ability to extrapolate. Most neural networks fail the moment the test data looks slightly different from the training data. SE-RRM was trained only on 9x9 Sudoku.

• Zero-Shot Scaling: It solved 4x4 puzzles with 95.46% accuracy without ever seeing a 4x4 grid. Prior models (HRM/TRM) scored 0%.
• Massive Grids: It achieved significantly higher-than-random accuracy on 16x16 and 25x25 grids—tasks that were physically impossible for its predecessors to even attempt.
• Data Efficiency: On the ARC-AGI benchmark, it matched SOTA results while using only 8 augmentations per sample, compared to the 1,000 augmentations required by TRM.

Critical Analysis & Future Outlook

SE-RRM proves that Efficiency > Scale. By using the right mathematical priors, a 2M parameter model can out-reason models 10x its size.

Limitations: The computational complexity increases with the number of symbols ($O(I^{2}K+K^{2}I)$). While efficient for Sudoku ($K=9$), it might become a bottleneck for tasks with thousands of unique symbols (like natural language vocabulary).

The Takeaway: For the next generation of AGI, we may need to move away from "learning everything from scratch" and toward architectures that inherently understand the geometric and symbolic symmetries of our world. SE-RRM is a major step in that direction.