PSRD: Mastering the High-Stakes Junctures—How Phase-wise Self-Reward Slashing Hallucinations in LVLMs

TL;DR

Large Vision-Language Models (LVLMs) often "hallucinate" details that aren't in the image. This paper identifies that these errors don't happen randomly—they peak at the onset of new semantic phases. The authors introduce PSRD (Phase-wise Self-Reward Decoding), a framework that uses a lightweight, self-distilled reward model to catch and correct hallucinations at these critical moments. No human tags, no massive fine-tuning, just smarter decoding.

Background: The Dynamic Nature of Error

Most current methods treat hallucination as a static problem: either you fine-tune the whole model to be "better," or you apply a generic penalty during decoding. However, the authors discovered a fascinating "Phase-wise" pattern.

As shown in the analysis, the error rate spikes at the beginning of each new descriptive segment. This is the "critical juncture" where the model shifts its internal state to describe a new object or attribute, and where its alignment with the visual input is most fragile.

Figure 1: PSRD Framework Overview. It monitors the response online and provides targeted interventions at phase boundaries.

Methodology: The "Scout-and-Project" Strategy

PSRD works through two key innovations:

1. Uncertainty-Guided Reward Model: Instead of querying a heavy LVLM at every step (too slow!), they distill the model's internal confidence into a lightweight reward model (based on CLIP). This model is trained using the LVLM's own "self-evaluation" as weak supervision, weighted by uncertainty to filter out noise.
2. Scout-and-Project Decoding:
   • Scouting: The model "scouts" the top-K candidate tokens for the start of a phase.
   • Projecting: If the reward is low, it "projects" an intervention strength ($\alpha$) using a secant-style update (local finite-difference) to find a version of the sentence that satisfies the visual grounding threshold.

Figure 2: Distribution of reward scores. The proposed training (L_HC) achieves much better separation between hallucinated and grounded samples.

Experimental Battlefront

The results are striking across five major benchmarks (AMBER, POPE, MMHal-Bench, etc.):

• Drastic Reduction: LLaVA-1.5-7B's hallucination rate dropped by 50%.
• Efficiency vs. Effectiveness: PSRD allows a controllable trade-off. By adjusting the threshold ($au$), developers can choose between lightning-fast inference or maximum grounded accuracy.
• Cross-Model Generalization: The reward model distilled from LLaVA-1.5 even works to improve other models like InstructBLIP and LLaVA-Next, proving that "discrimination" logic is shared across architectures.

Table 1: PSRD consistently outperforms other post-hoc methods and even rivals models fine-tuned on expensive human data.

Critical Insight: The Asymmetry of Intelligence

The core takeaway of this research is a "process-oriented" perspective. Faithful generation is hard—it requires long-horizon synthesis. However, discrimination (identifying if a specific phrase matches an image) is structurally more tractable.

By decoupling these two—using the generator's own latent knowledge to build a specialized "critic"—we can achieve SOTA reliability without the "brute force" approach of massive retraining.

Conclusion

PSRD proves that we don't always need more data; sometimes, we just need to listen to the model's own uncertainty. By intervening only when the risk is highest (at the start of semantic phases), PSRD offers a scalable, inference-time solution for the next generation of trustworthy multimodal AI.