Predictive metrics leave the perceptual space unexamined
Image quality metrics from PSNR and SSIM to deep-feature methods such as LPIPS, DreamSim, PieAPP and DISTS have mainly been optimized for predictive accuracy. The paper argues that this leaves basic questions open: what perceptual space organizes diverse images, how a quality judgment is formed inside it, how many dimensions are required, and whether that structure differs between low-level and high-level judgments.
The paper's answer is a multi-dimensional observer model. Each image is mapped to a distribution in a latent perceptual space, and a judgment is modeled as comparing noisy samples drawn from those distributions. Because the latent space is constrained to sit a simple affine transformation away from neural representations in the primate retinal and ventral stream, the learned dimensions can be interpreted using known properties of visual processing rather than treated as an arbitrary embedding.
A noisy observer picks the sample closer to the reference
In the triplet two-alternative forced choice (2AFC) trials used by the main datasets, an observer sees a reference image r and two alternatives x0 and x1. The model represents each image x as an N-variate Gaussian Z(x) with mean µ(x) and covariance Σ(x); the covariance depends on the image, so different images can carry different amounts of perceptual ambiguity. On a trial, the observer draws one sample from each of the three distributions and picks the alternative whose sample is closer to the reference sample.
Distance is measured with a weighted Minkowski norm whose per-dimension weights αm compensate for the different natural magnitudes of perceptual channels, and whose exponent p ranges from separable dimensions at p = 1 to integral dimensions at p = 2. Because the samples are noisy, the same observer can answer differently across trials and different observers can disagree; what the model predicts is the probability that x0 is chosen over x1, not a single deterministic answer.
The mean field is produced by a linear projection of an image encoding B(x) into N dimensions, which lets the model vary the intrinsic dimensionality without committing to a fixed value. The covariance field is generated through a Cholesky decomposition with a small MLP and a softplus on the diagonal, so that Σ(x) stays positive semi-definite by construction. The model is fit by maximum likelihood to behavioral data from standard triplet 2AFC experiments.
The model approaches human agreement on three 2AFC datasets
The evaluation covers BAPPS and PieAPP, where triplets differ through low-level distortions such as blur, noise and compression, and NIGHTS, where variation comes from the stochasticity of text-to-image generation rather than explicit distortions. On BAPPS, the CORnet-S-Multi model reaches agreement of .688 against a ceiling of .797 ± .183, and on PieAPP it reaches .710 against a ceiling of .766 ± .105. On NIGHTS it reaches accuracy of .859 against a ceiling of 1.000 ± 0.
These numbers place the observer model on par with or better than the compared perceptual metrics, but the comparison is not uniform across metric types. Only the observer model, CVVDP and PieAPP can output choice probabilities in 2AFC tasks, so KL divergence is reported only for those; DISTS, TOPIQ-FR and MILO predict quality scores as DMOS/MOS, LPIPS, DreamSim, OpenCLIP, DINOv3 and CLIP predict distances in a learned embedding space, and SSIM and PSNR are parameter-free classical metrics. Agreement, accuracy and KL divergence therefore answer different questions and should not be read as one scale.
Low-level quality judgments saturate within about ten dimensions
The central result comes from varying N, the dimensionality of the latent perceptual space, and finding where model performance stops improving. For low-level tasks, where quality is dictated by distortions such as blur and compression artifacts, performance generally saturates within 10 dimensions. That is far below the dimensionality of the image space, even after accounting for the sparsity present in natural images.
The paper also reports cross-domain fits that mirror this pattern. Training on BAPPS and testing on PieAPP, the CORnet-S-Multi encoder reaches agreement of .707 with a saturation point n* of 14 dimensions; training on BAPPS and testing on NIGHTS reaches accuracy of .825 with n* of 48; training on NIGHTS and testing on BAPPS reaches agreement of .670 with n* of 24. The paper notes that n* on the target dataset is often higher than its in-domain counterpart, so the saturation dimensionality is not a fixed property of the model alone.
Semantic-level judgments need an order of magnitude more dimensions
For high-level tasks that require semantic-level similarity processing, such as NIGHTS, the model saturates at a dimensionality an order of magnitude higher than the roughly 10 dimensions sufficient for low-level distortion tasks. The exact structure of the space, including its dimensionality and the information each dimension encodes, therefore varies between low-level and high-level quality judgments.
The cortical profile of the learned space tracks that difference. For low-level tasks, adding information from higher-order cortical regions from V1 and V2 up to V4 produces only a slight performance improvement, and including signals from area IT in fact degrades the model's performance. For high-level tasks, activity from IT markedly surpasses that from V1, so the high-level judgments clearly benefit from information originating in high-level cortical areas. The paper reads this as evidence that, despite a shared retinal encoding at the start, humans selectively construct task-dependent perceptual spaces in visual decision making.
Learned dimensions split into texture, color and object content
To interpret the dimensions, the paper uses the BRODEN concept annotations to characterize what each CORnet-S channel detects, then aggregates those meanings through the learned projection matrix. V1 channels predominantly select for color and low-level texture, V2 is dominated by texture, V4 begins producing parts and object fragments such as wheels, heads and ears, and IT is dominated by whole objects.
At N = 2, the recovered axes are readable. On PieAPP, one axis runs from large-scale repetitive structure such as architectural corridors to fine cluttered texture, while the other runs from warm, colorful outdoor scenes to muted indoor objects. On NIGHTS, one axis tracks scene multiplicity, from single foreground subjects to multiple repeating elements, and the other tracks color, from more colored images to near-monochromatic ones. At higher N the dimensions become more diverse and mixed: BAPPS and PieAPP concentrate weight on V1, V2 and V4 and remain dominated by color and texture, whereas NIGHTS develops substantial dependence on IT and reaches part- and object-level semantics. The paper describes higher dimensionality as a finer partition of the same hierarchy into specialized subspaces rather than redundant repetition.
The scale of the latent space limits cross-image comparison
Because the mean and covariance fields are learned jointly, the choice probability of a triplet stays the same if the mean and covariance of each image are scaled proportionally. That makes it difficult to compare distributions between images that do not appear in the same triplet. The paper suggests that more direct observations of the mean field, for example neural recordings collected during image quality assessment tasks or integrated cross-content judgments, would be needed to address this.
The paper also positions its model relative to the Bradley-Terry model, a classical psychometric scaling method that assumes perceived quality follows a Gumbel distribution. The two give similar predictive power, and the paper describes its observer model as a low-level interpretation of that phenomenological model. The practical reading is therefore not that a single dimension count describes human quality perception, but that the relevant count depends on the task: roughly 10 dimensions for distortion-driven judgments, an order of magnitude more when semantic similarity is at stake, with the cortical source of the useful information shifting from early areas toward IT as the task becomes more semantic.
Sources used in this answer
Multidimensional Observer Model and Perceptual Dimensions of Human Image Quality Assessment
Human image quality judgments are accounted for by extremely low-dimensional perceptual spaces, with low-level distortion tasks saturating within about 10 dimensions and semantic-level tasks requiring a dimensionality an order of magnitude higher.
