What’s changed: from black-box scores to interpretable, multi-dimensional evaluation
Older reward models often gave a single overall score with no explanation, which could hide biases and make it hard to know why a video was judged good or bad. Recent work has shifted toward breaking quality into specific dimensions—like motion, visual quality, and audio-visual alignment—so the evaluation is both more accurate and more transparent. VisionReward uses a hierarchical framework to capture fine-grained human preferences and then linearly weights these dimensions, making the scoring interpretable [1]. This approach beat existing models on both machine metrics and human evaluation, and notably improved preference prediction accuracy by 17.2% over VideoScore [1].
Similarly, McSc trains a generative reward model to decompose preferences into per-dimension assessments using self-critic reasoning chains, which helps the model understand the logic behind human judgments rather than just mimicking a score [2]. This shift is important because it addresses the 'black-box' problem: you can now see which aspect of a video (e.g., motion vs. visual quality) drove the score, and you can correct for conflicts between dimensions, like when high motion hurts visual quality [2].
How subjective quality is actually measured: human ratings, then models that predict them
The gold standard for subjective quality is collecting human ratings—mean opinion scores (MOS)—from many people watching and listening to videos. For audio-visual content, this means capturing both visual and audio quality together, because people’s overall experience depends on both. For example, the SJTU-UAV database collected MOS for 520 user-generated audio-video sequences, showing that audio quality significantly affects perceived quality [4]. Similarly, the OAVQAD+ database includes 625 distorted omnidirectional (360-degree) audio-visual sequences with MOS, making it the largest such database for VR content [3].
Once you have human ratings, you train a reward model to predict those ratings for new videos. The challenge is that human judgment is subjective and multifaceted, so a single score is insufficient. That’s why modern reward models like VisionReward and McSc use multi-dimensional scoring, and why some models are trained on both image and video tasks to improve generalization—UnifiedReward, for instance, learns to assess multiple vision tasks together, which improves both understanding and generation assessment [5].
Practical trade-offs: what works, what doesn’t, and what to watch out for
One major trade-off is between accuracy and interpretability. Fine-grained, multi-dimensional models like VisionReward are more interpretable and more accurate, but they require more detailed human annotations to train. McSc addresses the cost of human annotations by using self-critic reasoning to generate training data, reducing reliance on expensive human labels [2]. However, this introduces a risk: if the model’s reasoning is flawed, the reward signal could be biased.
Another issue is that some reward models may overfit to certain content types. For example, McSc found that aligning directly with overall preference can bias models toward low-motion content, so they added a motion-corrective re-weighting to mitigate this [2]. This highlights that subjective quality is not just about matching average human opinion but also about balancing conflicting dimensions.
Finally, audio-visual quality assessment is still an emerging field. Most existing databases and models focus on visual quality alone, but the papers here show that audio matters—for user-generated content and VR alike [3][4]. If you’re building a reward model for video-audio generation, you need to include audio in your human preference data, or you’ll miss a key part of the experience.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2023 to 2026, 4 from 2024 or later, 2 in Q1 journals, collectively cited 218 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 40 papers retrieved from a database of over 500 million.
Sources used in this answer
VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation
VisionReward introduces a hierarchical, multi-dimensional framework for image and video preference learning, achieving 17.2% higher preference prediction accuracy than VideoScore and a 31.6% higher pairwise win rate for text-to-video models using it as a reward model.
McSc: Motion-Corrective Preference Alignment for Video Generation with Self-Critic Hierarchical Reasoning
McSc uses a three-stage reinforcement learning framework with self-critic dimensional reasoning and hierarchical comparative reasoning to align video generation with human preferences, and adds motion-corrective DPO to avoid bias toward low-motion content.
Subjective and Objective Audio-Visual Quality Assessment for Omnidirectional Videos
The OAVQAD+ database includes 625 distorted omnidirectional audio-visual sequences with MOS, and the proposed OmniAVNet outperforms benchmark models on audio-visual quality assessment for VR content.
Subjective and Objective Audio-Visual Quality Assessment for User Generated Content
The SJTU-UAV database contains 520 in-the-wild user-generated audio-video sequences with MOS, and the proposed AVQA model outperforms benchmarks by jointly learning audio and visual features in the temporal domain.
Unified Reward Model for Multimodal Understanding and Generation
UnifiedReward is a unified reward model for multimodal understanding and generation, trained on a large-scale human preference dataset, and shows that jointly learning to assess diverse visual tasks yields mutual benefits.