Do people grade belief by ranks of disbelief rather than by probability?

Ranks ask for surprise, not a percentage

Ranking theory, as developed by Wolfgang Spohn, treats belief as graded by ranks of disbelief rather than by probability. A participant assigns a non-negative integer to a proposition: 0 if the proposition would not be surprising, a larger number if it would be more surprising, and infinity if it is impossible. The scale is ordinal and open-ended, so the same person can place a new proposition relative to earlier ones without rescaling everything to a fixed 0-to-1 range.

The paper's practical claim follows from that design: if you want to know what someone believes and how strongly, ask them for ranks. The authors argue this is computationally lighter than probability, because a ranker mainly has to order propositions and pick an intermediate number, whereas a probability reporter must keep the whole space of possible events internally consistent. That argument is the authors' interpretation of the task demands, not a separately measured result.

Ranks alone describe disbelief. To also express belief, the study used the theory's two-sided rank function, which compares the rank of a proposition with the rank of its negation. This is the device that lets one number line distinguish believing a statement, disbelieving it, and suspending judgment about it.

Six experiments spanning vague and known probabilities

The evidence comes from six experiments with University of Waterloo undergraduates. Experiment 1 was online with 168 students (median age 20; 101 female, 66 male, 1 undisclosed). Experiment 2 moved to an in-lab setting in a later semester with 63 students (median age 19; 48 female, 13 male, 2 who preferred not to answer). Participants finished in under 30 minutes on a custom JavaScript platform.

The propositions covered personal experience, chance devices, sequential learning, and an Ellsberg urn. In Experiments 1 and 2 the scenarios were everyday ones with unclear objective probabilities: whether a Quebec native speaks only one language, September daytime highs in Boston, and which music genres a cowboy-hat-wearing co-worker would enjoy at a karaoke party. Each set included a disjunction, such as Spanish or Urdu, or Jazz or Opera.

After each questionnaire, participants also reported subjective probability by filling a 10x10 grid of event cards, an open sampling format intended to capture probability without directly prompting numerical percentages. Explicit percentage estimates were collected only at the end, to avoid priming disbelief ranks. The analysis used non-linear least squares regression to fit a logarithmic base relating ranks to probabilities, with an inverse hyperbolic sine transform on ranks to limit the influence of outliers.

Two-sided ranks separate belief from suspension

Ranks were reliable and, through the two-sided rank function, distinguished belief, disbelief, and suspension of judgment. The authors report substantial concordance between the online and lab cohorts, which they read as evidence that the measure is not especially noisy. They also note that participants found generating ranks moderately difficult but clearly understood the task.

The comparison the authors draw is with explicit subjective probability reports, which they describe as noisy even with the graphical grid intended to make them easier. That contrast is the paper's own reading of its data; the packet does not give a single head-to-head noise statistic for the two measures.

The most consequential structural test was the Law of Negation: a person should never disbelieve both a proposition and its complement. Participants respected it in every task except a lottery. In the lottery case many felt free to disbelieve, to some degree, both that they could win and that they would lose, which the authors flag as a departure from the deductive coherence that philosophical lottery arguments assume.

Disjunctions are averaged, not minimized

Ranking theory's Law of Disjunction says the rank of "A or B" should be the minimum of the ranks of A and B. Participants did not do this. Instead they averaged the ranks of the disjuncts, a pattern the authors describe as the main systematic failure of the theory as a descriptive model.

This matters because the disjunction rule is where ranks diverge most sharply from probability. Probabilities of mutually exclusive events add, while ranks are supposed to combine by minimization; the paper notes that a logarithmic translation between ranks and probabilities can be made to work only with an infinitesimally small base, which the authors regard as strained. The averaging result therefore marks a specific, replicable place where human combination departs from both the normative rank account and the additive probability account.

The disjunction tests were embedded in the same everyday questionnaires as the other propositions, so the finding rests on judgments about language, weather, and music preferences rather than on abstract logic problems alone.

In learning, ranks track objective odds more closely

In a sequential learning task, participants played a computerized version of hide-and-seek in which the objective probabilities could actually be learned. The authors infer that participants did learn them, both because they won more games than chance when they could and because a matched distribution fit most participants' data well.

In this task ranks aligned more closely than subjective probabilities with the objective probabilities, and it is here that the paper finds its best evidence for the theoretically predicted logarithmic relation between ranks and probability. The authors are careful about scope: they do not conclude that people fail to represent environmental statistics, only that such representation is not the only, and perhaps not the primary, way people build and update degrees of belief.

The logarithmic fit is best supported precisely where objective probabilities are easy to detect. That condition limits how far the result travels to the vague everyday scenarios, where the authors describe the rank-probability relationship as non-linear, contextually variable, and noisy.

The Ellsberg paradox shrinks when measured by ranks

In the Ellsberg urn scenario, the paradoxical behavior seen with betting was markedly reduced when the same choices were computed from ranks. Ambiguity aversion measured from negative ranks was 8.62% in the one-urn condition and 5.17% in the rarity-urn condition. A randomization analysis that shuffled ranks within each proposition 1,000 times produced mean ambiguity-aversion rates of 21.15% and 19.98% respectively, so the observed rates sat away from the randomized central tendency.

The betting data themselves moved too: a chi-squared test for independence showed a significant decrease in participants exhibiting ambiguity aversion over four betting rounds (chi2(1) = 17.93, p < .001), and the comparison between ranks and bets was also significant (chi2(1) = 45.53, p < .001). Betting choices and disbelief were not significantly correlated (r = 0.07, p = .264, 95% CI [-0.05, 0.18]).

A two-way ANOVA found no difference in disbelief scores across propositions (F(3, 1996) = 0.003, p = 1) but a significant effect of urn scenario (F(1, 1996) = 5.206, p = 0.023), with higher overall disbelief in the rarity-urn scenario (median 98) than the one-urn scenario (median 3). The authors read the reduced ambiguity aversion as evidence that a within-participant rank assignment yields less paradoxical behavior, not merely as a quirk of how numbers were attached to these particular propositions.

What the rank account leaves open

The central limitation is built into the headline result: ranking theory survives as a description of the bipolar, negation-respecting structure of belief, but not as an account of how disjunctions combine. Any use of ranks to model compound propositions needs the averaging rule the participants actually used, not the minimization rule the theory specifies.

The samples are undergraduate students at one university, tested in online and lab sessions of under 30 minutes, and the everyday scenarios were chosen because their objective probabilities are unclear. The strongest logarithmic correspondence appears in the hide-and-seek learning task where probabilities were learnable, so the fit should not be assumed for the vague scenarios. The lottery is the one task where the Law of Negation failed, which limits claims about deductive coherence in lotteries specifically.

The authors also resist reducing ranks to probability. They note that a translation is possible in principle through a logarithm with an infinitesimal base, but they share Spohn's resistance to treating ranks as purely probabilistic and argue that belief cannot always be represented probabilistically in a consistent or deductive way.

A zero rank is not a 0.5 prior

The paper's most distinctive comparison is between the neutral points of the two frameworks. On the two-sided rank function, a rank of zero means no belief in either of two outcomes, such as whether a flipped coin will or will not be heads. The authors contrast this with the Bayesian neutral position, a prior of 0.5, which they describe as generally very strongly held.

That difference is the paper's own framing of what the rank measure buys: a way to record genuine suspension of judgment rather than encoding it as a maximally uncertain probability. Combined with the finding that ranks were reliable and less variable than subjective probabilities, it is the reason the authors propose ranking theory as a practical measurement tool for studies of belief and belief updating, while flagging disjunction averaging as the rule that must be revised before the theory can serve as a full descriptive model.

Sources used in this answer