A perception-only task with no predictive objective
The paper asks why a perceptual system would spend resources predicting what comes next. Its test is deliberately narrow: single-layer gated recurrent networks were trained to denoise noisy observations of musical tokens, a task the authors use as a metaphor for perception. The training loss concerned only the current latent token; nothing in the objective rewarded anticipating the future.
The data were 700 unique Bach compositions, drawn from 1,432 MIDI files and split by catalogue number into 490 training, 70 validation and 140 test compositions, so no movement of a test composition appeared in training. Each composition was converted into an event-based sequence of 12-dimensional binary pitch-class tokens, and observations were made by adding Gaussian noise. Fourteen noise amplitudes ran from nearly noiseless (σ = 0.001) to σ = 2, and network sizes ranged from 8 to 256 hidden units, with five independent initialisations per combination, giving 420 networks.
Freezing the network and probing its states
After denoising training, the recurrent weights and the denoising readout were frozen. A second linear readout was then fitted to predict the next latent token from the hidden state, so the network's internal dynamics stayed exactly as denoising had shaped them. If this readout carried predictive information, the frozen states themselves must contain it.
The comparison that matters is against benchmark models, not against zero. A liberal first-order Markov model saw the same noisy observation as the network; a conservative Markov model saw the clean current token, equivalent to a readout on perfectly denoised input. The authors reasoned that a network with genuine predictive information should beat the conservative benchmark, while a network without it should fall between the two. A separate RNN trained end-to-end on prediction served as an empirical ceiling. Because binary cross-entropy measures error, lower values mean better prediction.
Predictive information appears at moderate noise
In the case-example network, a 64-unit RNN trained at σ = 0.25, the prediction readout beat both Markovian benchmarks. Against the conservative benchmark, which receives the ground-truth current token, the effect was Cohen's d = 0.45 with p < 10⁻²⁰. That is the key result: the frozen denoising states held information about the upcoming token that was not simply derivable from the previous denoised input. As expected, the network still trailed the prediction-trained RNN (d = −0.64, p < 10⁻²⁰).
The noise dependence was sharp. The 64-unit network performed comparably to the conservative benchmark for σ ≤ 0.1, outperformed it for 0.15 ≤ σ ≤ 0.4, and fell behind it at higher noise. It beat the liberal benchmark, which receives the identical noisy input, for σ ≥ 0.1. The authors place the emergence point between σ = 0.1 and σ = 0.15, and report the pattern consistently for networks with at least 32 units.
State updates and states both track prediction error
The paper then checks two neural signatures often treated as evidence for predictive processing. First, it asks whether the size of the state update correlates with prediction error after controlling for how much the latent token actually changed. In the case-example network, the partial correlation was positive, and the effect held across network sizes and noise regimes, though some cells were not significant after Bonferroni correction over 84 comparisons.
Second, it asks a stronger question: can prediction error itself be decoded from the hidden state? Ridge regressions were fitted on the validation set and evaluated on the 140 test compositions, with the most stringent benchmark decoding error from the current and previous observations. The case-example readout beat that benchmark in all compositions (d = 3.46, p < 10⁻²⁰). Robust encoding appeared for 64-unit networks at 0.15 ≤ σ < 0.6, the same band where the networks beat the Markovian benchmarks, and was not apparent below 32 units.
Why only the middle noise band supports prediction
The authors explain the band using detection theory. Assuming each of the 12 latent dimensions is an independent fair coin flip, the probability of recovering the latent exactly from one observation is above 0.99 when σ < 0.15. At that point the instantaneous observation is already almost always sufficient, so a generative model adds little. Above σ = 0.4, that probability falls below 0.27, single observations become individually uninformative, and the observations no longer support learning the statistical structure prediction would need.
This is the paper's interpretation, not a directly measured mechanism, and it is consistent with filtering theory as the authors describe it: passive decoding suffices in a nearly noiseless channel, marginal statistics dominate in a very noisy one, and prediction pays only in between. The practical consequence is a qualitative hypothesis rather than a fixed law: predictive processing should be expected in systems shaped to operate under moderate noise, not necessarily in every sensory population.
What the simulation does and does not establish
The evidence comes from simulations on tokenised Bach, not from neural recordings. Music was chosen because it is generated by a structured system complex enough that prediction is possible and not trivially first-order Markovian, but the results are still a model of perception, and the authors present them as a testable qualitative hypothesis. The noise levels are Gaussian perturbations of binary pitch-class tokens, and the networks are single-layer GRUs, so the findings do not by themselves transfer to other architectures, sensory modalities or real brains.
Two further boundaries sit close to the results. The predictive readout is linear and fitted after training, so it measures information available in the frozen states rather than a prediction the network was optimised to produce. And the prediction-error decoding was robust only in a restricted band of noise and network size, with no apparent encoding below 32 units. The paper's own discussion extends the hypothesis to individual differences in sensory noise and to conditions involving over- or under-reliance on prediction, but those extensions are proposed, not tested here.
Task demands, not architecture, decide prediction
The paper's distinctive claim is a shift in what explains predictive processing. Earlier work cited in its discussion had shown prediction emerging in RNNs under energy constraints; here the authors argue that no predictive or metabolic constraint and no presupposed architecture were needed. What mattered was the difficulty of the perceptual task, indexed by the noise regime in which the network was trained.
That makes the noise band, not the presence of recurrence alone, the variable to watch. A denoising-trained network at σ = 0.25 encoded predictions and prediction error; the same class of network at very low or very high noise did not show a demonstrable capacity to predict. For readers weighing claims about predictive processing, the paper's priority is therefore to ask under what noise and capacity conditions a system was shaped, and to treat prediction as an emergent property of that regime rather than a default feature of perception.
Sources used in this answer
Prediction emerges in RNNs trained for perception
Denoising-trained RNNs encoded predictive information and prediction error without a predictive objective, mainly at intermediate noise levels (0.15 ≤ σ ≤ 0.4).