[CVPR 2025] VGPO: Mitigating Temporal Forgetting in Multimodal Reasoning via Visual Focus Compensation

Executive Summary

In the race to build more powerful Vision-Language Models (VLMs), the community has successfully improved logical reasoning through Reinforcement Learning with Verifiable Rewards (RLVR). However, a silent failure mode persists: as the model "thinks" longer, it begins to "see" less. This phenomenon, termed Temporal Visual Forgetting, results in models ignoring the image in favor of textual priors during the final, critical steps of a proof or calculation.

Visually-Guided Policy Optimization (VGPO) addresses this by turning the model's internal representations into a guiding signal. Without needing auxiliary models or expensive external verifiers, VGPO uses hidden state similarities to "remind" the model to look back at the image, achieving SOTA performance across major benchmarks like MathVista and LogicVista.

The "VLM Amnesia": Why Long Reasoning Fails

The authors identified three critical flaws in current multimodal reasoning:

1. Textual Dominance: VLMs naturally favor text history over visual tokens.
2. Attention Decay: Visual attention peaks early and then plummets as the reasoning chain extends.
3. Correctness Correlation: Successful trajectories maintain a higher "late-stage" visual attention ratio compared to failed ones.

Existing solutions often rely on "Look-Back" tokens or external GPT-4/5 verification, which adds inference overhead. The core insight of VGPO is that the information required to fix this is already inside the model's own hidden states.

Methodology: The VGPO Framework

The framework operates through a three-stage pipeline that transforms internal states into explicit optimization incentives.

1. Internal Visual Focus Score

Instead of using heavy attention maps, VGPO computes the cosine similarity between the current generated token's hidden state and a global visual prototype (mean-pooled image tokens). This effectively identifies which tokens are "grounded" in the image.

2. Visual Attention Compensation (VAC)

Since visual attention naturally decays, VGPO applies a linear compensation schedule. As the step count $t$ increases, the importance of visual focus is amplified: $w_{i,t}={\rho }_{i,t}\cdot [1+extGate\cdot \beta \cdot \frac{t}{T}]$ This ensures that grounded tokens at the end of a chain receive a stronger optimization signal than those at the beginning.

3. Dual-Grained Advantage Re-weighting

VGPO modifies the standard GRPO (Group Relative Policy Optimization) objective:

• Intra-trajectory: Incentivizes individual tokens within a path that show high visual focus.
• Inter-trajectory: Rewards entire "thought paths" that maintain consistent visual faithfulness compared to others in the same group.

Experiments & SOTA Results

VGPO was tested on a suite of "Vision-Dependent" benchmarks (where you can't guess the answer from text alone) and general mathematical reasoning.

• Efficiency: The 7B VGPO model achieved 66.6% on Avg-Math, rivaling the performance of the closed-source Qwen2.5-VL-72B model.
• Stability: Unlike standard RL methods that suffer from reward oscillation, VGPO's training curve is significantly smoother, proving that visual grounding acts as a regularizer.
• Scaling: Performance gains were consistent across 3B, 7B, and 32B model scales.

Deep Insight: Is "Looking Harder" Enough?

An interesting finding in the paper's failure analysis is that while VGPO forces the model to look, it cannot fix a bad "eye." If the visual encoder misinterprets a blurry number (e.g., seeing a '6' as an '8'), VGPO will actually reinforce the error by making the model more confident in its visual misperception. This reveals that the next frontier is not just policy optimization, but scaling the resolution and representational quality of the visual backbone itself.

Conclusion

VGPO provides a elegant, "zero-external-dependency" way to keep VLMs honest. By compensating for the natural mathematical decay of attention, it bridges the gap between seeing and thinking—a crucial step for reliable AI in STEM and complex reasoning domains.

Takeaway for Practitioners

If your VLM is failing on long reasoning tasks, don't just add more text data or larger rewards. Check the late/early visual accumulation ratio. Often, the model simply "forgot" what it was looking at halfway through the problem.