[CVPR 2026] VGPO: Combating Temporal Visual Forgetting in Multimodal Reasoning

TL;DR

Vision-Language Models (VLMs) often "forget" the image as they write long explanations. Visually-Guided Policy Optimization (VGPO) fixes this by using a model's own hidden states to refocus on visual cues during Reinforcement Learning. It achieves state-of-the-art results (33.2% gain on math reasoning) by essentially "reminding" the model to look back at the picture more intensely as the reasoning steps progress.

The Problem: Text-Dominance and Visual Forgetting

Despite the "Vision" in VLM, most models are heavily biased toward their LLM roots. During inference, they exhibit two fatal flaws:

1. Sparse Visual Activation: Models often take a quick "glance" at the image and then ignore it in favor of textual priors.
2. Temporal Visual Forgetting: As the reasoning chain gets longer, the attention weight on visual tokens decays. The authors found a clear correlation: correct reasoning paths maintain a higher "Late/Early" visual attention ratio than incorrect ones.

Figure 1: Empirical evidence shows that visual attention (red line) significantly decays as the number of generation steps increases.

Methodology: VGPO - Looking Closer, Longer

The core innovation of VGPO is that it doesn't need a "teacher" model (like GPT-5) to tell it where to look. It uses Visual Focus Scores derived from the cosine similarity between the current token's hidden state and a "visual prototype" (the mean of all image tokens).

1. Visual Attention Compensation (VAC)

Because visual attention naturally decays, VGPO introduces a linear compensation schedule. As the time step $t$ increases, the reward signal for visually-grounded tokens is progressively amplified. This acts as a "counter-decay" force.

2. Dual-Grained Advantage Re-weighting

VGPO modifies the advantage function in Reinforcement Learning (specifically building on GRPO) at two levels:

• Intra-trajectory: Incentivizes specific tokens within a sentence that show high visual similarity.
• Inter-trajectory: Rewards entire reasoning paths that maintain consistent visual grounding compared to other paths in the same group.

Figure 2: The VGPO architecture: From hidden state similarity to dual-grained advantage re-weighting.

Experiments & Results

VGPO was tested across a massive suite of benchmarks including MathVista and MMMU-Pro.

• Efficiency: The Qwen2.5-VL-7B model equipped with VGPO rivaled the performance of much larger models (72B).
• Ablation: The study confirmed that both intra- and inter-trajectory re-weighting are essential. Removing either leads to a drop in performance, particularly in vision-dependent tasks like object counting.

Table 1: VGPO achieves SOTA results, significantly outperforming DAPO and vanilla GRPO.

Critical Insights: Why it Works

The brilliance of VGPO lies in its Self-Guidance. While previous SOTA methods (like PAPO or VPPO) required noisy image rollouts or auxiliary models to verify visual faithfulness, VGPO proves that the model already knows which tokens are visually grounded—it just needs the policy optimization to prioritize them.

Limitations

• Encoder Dependency: If the initial visual encoder is poor (e.g., at low resolution), VGPO will simply reinforce the model's "confident" misinterpretation of the image features.
• Heuristic Nature: The linear compensation is a strong heuristic, but it might not be optimal for cases where the final steps of a problem are purely symbolic (e.g., pure mental arithmetic after the geometry is parsed).

Conclusion

VGPO represents a significant step toward "true" multimodal reasoning. By addressing the temporal decay of visual attention, it ensures that models remain grounded in the visual evidence from the first step to the final box. This framework is a must-watch for researchers looking to stabilize RL training in multimodal environments.