How do separators make carry propagation learnable in multiplication transformers?

The input format decides whether carry is learnable

The paper's central comparison is between two ways of writing the same problem. In the spaced format a space token separates consecutive digits, in MSB-first order, for example 0 1 2 3 * 0 4 5 6 = 0 0 0 5 6 0 8 8. The compact format removes those spaces. Both are trained on 1M zero-padded 4x4 multiplication examples with operands in [0,9999] and answers zero-padded to eight digits, answer-only, without chain of thought.

The gap is not marginal. The spaced format reaches 89% exact match (best of three seeds, which reach 89%, 80% and 78%), while the compact format reaches 1% in a single run. Tracking per-position error during training explains why: in the compact format the middle positions never converge, whereas in the spaced format all positions converge, from the outside in.

The authors offer a mechanism for this, which should be read as their explanation rather than a measured result: separators give each digit a prediction slot with no other job, while in the compact format the same hidden state must both represent a digit and predict the next one. The paper notes that with chain of thought the model reaches 100% in 13k steps, but the computation is then written in the output and there is nothing left to find inside the network.

Positions converge from the outside in

For each output position indexed LSB-first, the study measures per-position log error log(|p_i - g_i| + 1) on a held-out set of 2K examples throughout training. In the spaced format, positions converge in a consistent order across seeds: the outer positions (0 and 7) first, then 1 and 6, then 2 and 5, and the middle positions (3 and 4) last.

That order tracks the dependency structure of long multiplication. Positions 0 and 7 depend on a single input-digit pair, positions 1 and 6 on two, and position 3 on all four pairs in its column plus the largest carries. The paper draws a practical consequence from this: a model at intermediate exact-match accuracy is not halfway done, it has solved several positions cleanly and is still struggling on a few.

The same positions whose prediction slots cover the most of their ring, 3 and 4, are the ones learned last. This links the learning dynamics to the geometry described next.

The carry-in is an angle on a ring

On the answer side, the space tokens stop being neutral. Separated by output column, each prediction slot forms its own ring-shaped cloud in the leading principal components. Splitting examples by carry count k and fitting one PCA per group shows the two extremes: at k = 0 the prediction slots form short arcs, at k = 7 nearly closed rings, with intermediate k interpolating.

A diffusion map of the column-3 prediction slot identifies what the ring encodes. At block 3, points are ordered around it by their carry-in. This explains the dependence on k: examples with many carries take many different carry values and so fill a larger part of the ring, while examples with no carry all share a carry-in of zero and sit on a small part of it. The paper calls the ring a carry dial, with the carry count deciding how much of the dial is used.

The ring does not persist unchanged. At block 5 the ring points are no longer ordered by carry-in but group by output digit instead, as the slot commits to the digit it will emit. Separately, PCA of the input token embeddings shows digits 0-9 lying on a circle ordered by value after training, with the space token near the centre because as an input it carries no digit value.

Patching shows the state is causally used

Ring geometry alone is a correlation between prediction-slot states and carry-in. To test whether the model uses these states, the authors apply matched activation patching to output column 3 with source carry-in 5 and target carry-in 10. Each target example is paired with the individual source example whose column sum is closest, an exact match in 90-96% of cases across the columns tested, which holds fixed the only other quantity that determines the correct digit.

In blocks 0-1 the effect is small (0.05-0.11) and spread over input digits and answer positions. From block 2 it concentrates on the prediction slot of column 3: the fraction of examples whose digit becomes the one implied by the source carry reaches 0.46, 0.67 and 0.84 after blocks 2, 3 and 4, with every other position near zero. Replacing this single vector at an intermediate layer, while later layers can still read the rest of the sequence, is enough to make the model answer as if the carry were different.

The matching procedure matters for interpreting this number. Patching the unmatched source-group mean instead conflates the carry with variation in the column sum and gives roughly 0.10, against 0.85-0.95 with matching when all layers are patched for column 4 with carry 5 to 10. The paper also flags that patching at the last block is a sanity check rather than evidence, since after block 5 the prediction slot's state alone determines the output; evidence about where the carry is held comes from blocks 4 and earlier.

Boundary columns plateau early, interior columns late

Repeating layer-by-layer patching of each column's prediction slot for columns 1-7 across three seeds reveals two regimes. Column 1, whose carry-in depends on a single upstream product, and column 7, which has no partial product of its own, reach their plateau (0.78 and 1.00) by block 1 in all three seeds. For the interior columns 2-6, whose carry-in accumulates a chain of upstream columns, the effect rises only in later blocks, with an onset that varies modestly across seeds.

Varying the carry difference tests whether the slot holds the carry value. Patching only the prediction slot up to block 4, leaving the last block free to read the rest of the sequence, the column-3 digit follows the shift implied by the carry difference in 95-100% of cases for every difference tested, as do columns 1, 2 and 7. For columns 4-6 the shift is only partial, consistent with their carry still being held at the neighboring slot at block 4.

The route to the slot differs by column. For column 4, patching the column's own prediction slot has no effect up to block 3; at blocks 2-3 the effect sits instead at the prediction slot of the preceding, higher-order column (0.36-0.39), which also needs the column-4 carry-in to predict its own digit, before moving to the column-4 slot (0.15 after block 4). The carry is assembled at the neighboring answer slot and relayed, rather than accumulated in place.

Remaining errors are almost always off by one

The errors that remain at convergence concentrate at positions 3 and 4 and are almost always off by plus or minus one. The paper offers two readings, both pointing to the carry. By the digit equation, a carry estimated one unit too high or too low shifts the digit by exactly one. And since digits are represented on a circle, a small angular error also yields a neighboring digit.

This is a diagnostic rather than a separate result: it tells readers what the residual failure mode looks like once the model has otherwise learned the task, and it is consistent with the carry representation being the locus of the remaining imprecision.

What the causal claim does and does not cover

The patching results support a specific claim: a single site is sufficient to change the digit. They do not show that information is present only there. For columns 4-6 the carry is represented at the prediction slot, yet patching it alone transfers it only partially. Because pairs are matched on the column sum, a successful patch also cannot separate the carry from the column result plus carry; pairs whose column sums differ modulo 10 would be needed to do that. And patching locates where the carry is held, not how it is computed: the attention heads that perform the relay are not identified.

The scope is narrower than the framing might suggest. The main spaced configuration is trained with three seeds, but the compact configuration uses a single run, and the study uses a single model size and architecture: hidden size 256, 4 attention heads, intermediate size 704, 6 layers, 4.8M non-embedding parameters. All results concern 4x4 multiplication with one tokenization. The geometric analysis uses one seed per figure, and the dependence of the ring on carry count reflects how many distinct carry values a group covers, not a change in the encoding itself.

The authors position these findings as complementing rather than contradicting the bag-of-heuristics picture found in pretrained LLMs, on the grounds that those models were never trained on a format that gives the carry a dedicated slot.

The separator slot is the priority signal

The most decision-relevant split in this paper is not between models but between columns and between formats. Format first: the same architecture and data go from 1% to 89% exact match purely by inserting a space token between digits, so anyone training a transformer on multi-digit arithmetic should treat the input format as a first-order variable rather than a preprocessing detail.

Column second: boundary columns 1 and 7 reach their patching plateau by block 1, while interior columns 2-6 rise only in later blocks, and columns 4-6 hold their carry at a neighboring slot at block 4 rather than in place. If you are probing where a carry lives, the column you choose changes the answer, and the prediction slot immediately before each digit is the place to look first.

The residual error profile gives a concrete threshold for reading progress. Because remaining errors concentrate at positions 3 and 4 and are almost always off by one, an intermediate exact-match number reflects a model that has cleanly solved the outer positions and is still failing on the longest-range middle columns, not one that is uniformly partway there.

Sources used in this answer