How can learning asymmetry be distinguished from perseveration in behavioral models?

Two processes can produce the same choice repetition

Choice repetition is the observable behavior. The paper asks which of two hidden processes produced it. A learning asymmetry means an agent learns differently from outcomes depending on their valence, or depending on whether the outcome confirms the choice just made; a confirmation bias is the specific case where confirming outcomes are weighted more heavily than disconfirming ones. Choice perseveration is different: it is an outcome-independent tendency to repeat past choices, regardless of what those choices earned.

The distinction matters because the two processes make the same prediction about raw behavior. An agent that keeps picking the same option may do so because it learned that the option is good, or simply because it tends to repeat. Evidence for asymmetric learning has typically come from computational models that include both mechanisms at once and control for perseveration, or from specific behavioral markers. The paper argues that both routes have critical flaws.

A hybrid model can invent a bias that is not there

The paper's first demonstration uses simulation, which lets the authors know the ground truth. They simulate agents that have perseveration but strictly symmetric learning, then fit those simulated behaviors with a hybrid model that estimates both perseveration and a confirmation bias. The hybrid model systematically attributes learning asymmetry to agents that have none, because the two mechanisms partially mimic each other.

The mechanism is a tradeoff inside the fit. The model underestimates the level of perseveration, and to avoid under-predicting the observed rate of choice repetition it raises the estimated confirmation bias. Variance that belongs to a single perseverative mechanism is split into two misestimated mechanisms. The authors also report the reverse direction: simulating agents with a learning asymmetry but no perseveration and fitting them with the hybrid model produces a smaller spurious perseveration, so the mimicry is mitigated but not absent in that direction.

Shrinking priors inflate the spurious bias

The fitting procedure adds a second problem. The authors used maximum a posteriori (MAP) estimation, which combines the data with prior distributions over parameters. Following earlier work, the perseveration parameter was given a narrow normal prior centered at zero. Because perseveration is generally positive in these datasets, such a prior pulls the estimate downward.

A parameter recovery study quantifies the consequence. Simulated datasets were generated from the hybrid model and refitted with it, pooling across experiments for 509 participants. MAP underestimated the generative perseveration parameter (mean difference −0.288, 95% CI [−0.318, −0.258], t(508) = −18.826, d = −0.834, p < 0.001) and overestimated the confirmation bias (mean difference 0.030, 95% CI [0.015, 0.046], t(508) = 3.850, d = 0.171, p < 0.001). Widening the prior reduced the spurious confirmation bias, and both prior widths used in previous research still produced one.

Switching to maximum likelihood estimation (MLE), which is effectively MAP with non-informative priors, removed the significant underestimation of perseveration (mean difference −0.131, 95% CI [−0.486, 0.224], p = 0.468) and the significant error in confirmation bias (mean difference −0.013, 95% CI [−0.034, 0.008], p = 0.227). In empirical data, MAP gave lower perseveration estimates and higher confirmation bias estimates than MLE. Yet MLE did not eliminate the problem entirely: the authors note that a spurious bias is still detected, and that reaching an unbiased asymptotic regime may require an unfeasibly large number of trials.

The APPS test builds a null from each dataset

The proposed test keeps the hybrid model's asymmetry estimates but stops treating them as unbiased. It asks whether the observed group-level confirmation bias exceeds what would be expected if the true population mean learning asymmetry were zero, while perseveration is still present.

The construction uses paired artificial agents. Each participant's fitted parameters define an agent, and an anti-agent is identical except that the confirmatory and disconfirmatory learning rates are swapped. By construction, the average confirmation bias across each pair is exactly zero. Each of the 2N agents generates 1001 simulated behavioral datasets on the experimental task, and each dataset is refitted with the hybrid model using MLE. To build the null distribution, the procedure is repeated 10,000 times: each iteration randomly selects exactly one member from each agent–anti-agent pair and one of that member's simulated datasets, then averages the estimated confirmation bias across the selected N agents. The one-sided p-value is the proportion of these 10,000 null samples whose mean confirmation bias is greater than or equal to the empirical group mean.

Because the null is generated from empirical parameter estimates, it absorbs the estimation biases operating in that particular parameter range and task design. The authors report that learning asymmetries vary substantially with experimental design, such as the number of trials, and with participants' generative parameters. The cost is that the null distribution must be recalculated for every dataset rather than reused.

Type I error rates separate the fitting procedures

The paper compares error rates across fitting methods. With MAP, most datasets showed markedly inflated Type I error rates at the group level, as high as about 100%. MLE, whose estimation biases are considerably smaller, seemed to control the Type I error rate adequately in most but not all studies, with a maximal rate of about 11%. That is not a guarantee: when reinforcement-learning agents were sampled from parameter regimes producing especially strong recovery biases, MLE could also reach Type I error rates as high as about 100%.

The APPS test was more robust to those regimes, though not fully so, producing a Type I error rate around 12% in the case where the standard t test reached 100%. The authors also verified that the test adequately controlled the Type I error rate under typical experimental conditions and compared its statistical power with alternatives.

Some confirmation biases survive the stricter test

Applying the test to a large dataset spanning ten published experiments across four studies changes the picture. A robust confirmation bias is less widespread across experiments than previously thought: only 3 out of the experiments showed one. The authors describe some previously reported confirmation biases as fragile, while others remain robust even at the across-study level.

This is a statement about the evidence for a confirmation bias, not a claim that asymmetric learning is absent. The paper's discussion concludes that confirmation bias remains a robust explanatory mechanism within human reinforcement learning and cannot be reduced to a byproduct of choice perseveration, while the strength of evidence is more nuanced than earlier reports suggested.

Common behavioral signatures do not dissociate the processes

The paper also tests behavioral markers proposed as qualitative evidence for confirmation bias, simulating both a confirmation-bias model with no perseveration and a perseverative symmetric-learning model. In a task with two equally rewarding random bandits, each rewarding with 50% probability, a confirmation-bias model predicts a stronger preference for one option; the perseverative symmetric-learning model produces a similar pattern. In a reversal task, a confirmation-bias model predicts slower adaptation after contingencies reverse; perseveration produces similar effects again.

A regression-based signature fares no better: regressing the probability of repeating a choice on trial t+1 on choices from trials t and t−1 is also sensitive to perseveration. The authors conclude that these signatures cannot conclusively implicate a learning asymmetry, and that strong perseveration can even mask a true underlying confirmation bias. They therefore propose a task design intended to yield a more valid qualitative signature.

Which datasets need the stricter test first

The paper's practical priority follows from where the bias enters. The spurious confirmation bias is largest when perseveration is underestimated, which happened under MAP with a narrow prior and, to a lesser degree, under MLE in parameter regimes with strong recovery bias. A study that reports a confirmation bias from a hybrid model should therefore first check its parameter recovery in its own design and parameter range, then apply a null distribution recalculated for that dataset rather than relying on a fixed threshold.

The authors extend the caution to individual differences. They note that learning asymmetries have been proposed as signatures in mental health conditions, developmental change, and political ideology, and warn that studies deriving learning biases from choice behavior risk misattributing perseveration to asymmetric learning. Their parameter sweep also shows that the magnitude of the spurious asymmetry can be systematically affected by other parameters such as choice stochasticity, which complicates reading model parameters as proxies for specific cognitive mechanisms. The paper's own scope is confirmation bias in reinforcement learning with perseveration treated as a nuisance process; the authors note that biased estimation of perseveration appeared less problematic in this dataset, but that this can depend on task design and the sampled parameter range, and that their pipeline can be adapted to reverse the roles.

Sources used in this answer