Why human judgment is the ground truth—and why it's hard to automate
Subjective quality ultimately means what people perceive as good, and that can't be fully captured by any single automated metric. A 2022 study on video frame interpolation collected ratings from 60 human participants on 180 distorted videos and found that even the best-performing common metric, LPIPS, achieved only a Spearman rank-order correlation of below 0.6 with human opinions—meaning it explained less than 36% of the variance in perceived quality [5]. This tells you that for any video generation task, including world models, you need human studies to anchor what 'good' means.
The same principle applies to high dynamic range (HDR) video, where a 2023 study gathered over 20,000 human judgments across 310 videos to build a quality database [1]. That study showed that adding HDR-specific features to a standard quality model (VMAF) improved its correlation with human ratings, but the point is that the human data came first. For few-step world models, you should similarly collect subjective ratings—for example, by showing paired trajectories to viewers and asking them to rate visual quality and action plausibility—and then use those ratings to validate or calibrate automated metrics.
Which automated metrics to use—and how to read their numbers
For visual fidelity, the most useful reference-based metric is LPIPS (Learned Perceptual Image Patch Similarity), which measures perceptual distance between generated and reference frames. In the ForgeWM study on few-step action-conditioned world models, the best-performing system achieved the lowest reference LPIPS among evaluated systems, indicating that lower LPIPS corresponded to better perceived quality in that context [3]. However, because LPIPS alone is insufficient (as shown by the 2022 study), you should also report PSNR and SSIM, even though they are even less correlated with human perception [5].
For action-conditioned models, you must also evaluate whether the generated video correctly reflects the input actions. ForgeWM reported 'action-sign accuracy' and 'mouse-control accuracy' as separate metrics, and they also measured 'reference-aligned motion-profile agreement' to check that the motion matches the reference trajectory [3]. These are not standard video quality metrics, but they are essential for world models because a visually perfect video that ignores the action is a failure. So your evaluation suite should include: LPIPS for perceptual fidelity, action-alignment metrics (e.g., accuracy of predicted action effects), and human ratings for overall quality.
Don't forget interaction and viewing conditions—they change what 'quality' means
Subjective quality is not just about pixels; it depends on how the video is consumed. A 2023 chapter on volumetric video quality assessment emphasized that user interaction (e.g., viewpoint changes) and display settings (e.g., VR headsets vs. monitors) significantly affect perceived quality [4]. For few-step world models used in interactive settings (like games), you should evaluate quality under the actual viewing conditions—for example, with a gamepad or mouse, and on the target display—because a model that looks good in a static view may fail when the user interacts.
The ForgeWM study also highlights the importance of latency: they designed a 'dual-path deployment protocol' where a one-step student model handles latency-critical interaction, and a replay-time refinement step improves quality offline [3]. This suggests that subjective quality evaluation should distinguish between real-time interactive quality (where speed matters) and offline quality (where you can afford more compute). You might ask users to rate both the interactive experience and the final refined video, as they may value different trade-offs.
About These Sources
This answer is built on 5 studies (3 peer-reviewed, 2 preprints) — published from 2022 to 2026, 2 from 2024 or later, 1 in Q1 journals, collectively cited 54 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 35 papers retrieved from a database of over 500 million.
Sources used in this answer
A Study of Subjective and Objective Quality Assessment of HDR Videos
A 2023 study created the first public HDR video quality database (LIVE HDR) with 310 videos and over 20,000 human judgments, and showed that adding HDR-specific features (HDRMAX) to VMAF improved quality prediction on both HDR and SDR content.
Olaf-World: Orienting Latent Actions for Video World Modeling
Olaf-World (2026) introduces a sequence-level alignment objective (SeqΔ-REPA) that anchors latent actions to temporal feature differences from a frozen self-supervised encoder, improving zero-shot action transfer and data efficiency in action-conditioned video world models.
ForgeWM: Progressive Causal Training for Few-Step Action-Conditioned Video World Models
ForgeWM (2026) presents a progressive training pipeline for few-step action-conditioned world models, achieving the lowest reference LPIPS and best action-sign and mouse-control accuracy among evaluated systems on paired Minecraft trajectories, with a dual-path deployment for latency-critical interaction and replay-time refinement.
Subjective and objective quality assessment for volumetric video
A 2023 chapter on volumetric video quality assessment discusses how user interaction, display settings, and rendering parameters affect subjective quality, and reviews advantages/disadvantages of different subjective and objective methods.
A Subjective Quality Study for Video Frame Interpolation
A 2022 subjective study on video frame interpolation (BVI-VFI) collected ratings from 60 participants on 180 distorted videos and found that common metrics like PSNR, SSIM, and LPIPS all correlated poorly with perceived quality (best SROCC below 0.6), highlighting the need for bespoke perceptual metrics.