[CVPR 2026] PyVision-RL: Stopping "Interaction Collapse" to Forge Truly Agentic Vision Models
PyVision-RL: Forging Open Agentic Vision Models via RL
PyVision-RL is a reinforcement learning framework designed to forge open-weight multimodal agents that use Python as a dynamic tool for image and video reasoning. It introduces an oversampling-filtering-ranking rollout strategy and an accumulative tool reward to stabilize training and prevent interaction collapse, achieving SOTA results across various vision-agent benchmarks.
TL;DR
PyVision-RL is a breakthrough RL framework for open-weight multimodal models (like Qwen2.5-VL) that enables stable, multi-turn tool-using behavior. By treating Python as a primitive tool and implementing a novel rollout ranking system, it achieves SOTA performance in visual math and search. Most impressively, its PyVision-Video model achieves higher accuracy in spatial reasoning using 9x fewer visual tokens than traditional uniform sampling methods.
The Core Challenge: The Collapse of Interaction
In the quest to move from passive MLLMs to active "agents," Reinforcement Learning (RL) is the weapon of choice. However, researchers have hit a wall: Interaction Collapse. During RL, models often discover that the easiest way to maximize rewards is to stop using tools. Tools introduce complexity and potential points of failure; thus, models "collapse" into short-cut behaviors, ignoring the very tools (like zooming or cropping) that make them powerful.
PyVision-RL posits that this isn't a limitation of the models, but of the training incentives and the rollout selection process.
Methodology: High-Efficiency Agentic Scaffolds
The researchers introduced two distinct scaffolds for images and videos, unified by a Python execution sandbox.
1. Python as a Primitive Tool
Instead of a fixed menu of tools (e.g., crop_tool, zoom_tool), the model generates raw Python code. This "Dynamic Tooling" allows the model to calculate pixel statistics, rotate images, or fetch specific video frames algorithmically.
2. The RL Stability Trifecta
To solve the stability issues found in vanilla GRPO (Group Relative Policy Optimization), PyVision-RL introduces:
- Accumulative Tool Reward: Models get a bonus for every tool call, but only if the final answer is correct. This prevents "spamming" tools while encouraging deep reasoning.
- Standard Deviation Sorting: Not all training samples are equal. This strategy selects prompts with high reward variance—meaning they are neither too easy nor too hard—creating an organic curriculum.
- On-Demand Context Construction: Traditionally, videos are sampled at 1 FPS, flooding the model with 45K+ tokens. PyVision-Video starts with zero frames in the context, using Python to "fetch" only the timestamps it deems relevant.
Figure 1: The agentic scaffolds of PyVision-RL for image and video tasks.
Experiments: More Accuracy, Fewer Tokens
Visual Search and Reasoning
On the V* benchmark (visual search) and WeMath (complex mathematical reasoning), PyVision-Image demonstrated a clear lead over existing methods. By iteratively zooming into high-resolution images, the model catches details that standard models miss.
| Model | V* (Visual Search) | WeMath (Math Reasoning) |
|---|---|---|
| Qwen2.5-VL-7B | 78.5 | 34.6 |
| DeepEyes-v2 | 81.8 | 38.1 |
| PyVision-Image | 88.7 | 47.7 |
The Video Efficiency Miracle
The most striking result is the efficiency of PyVision-Video. By sampling frames "on-demand," it outperformed the base Qwen2.5-VL while using a fraction of the compute.
Figure 4: The efficiency-performance trade-off on VSI-Bench. PyVision-Video (top left) achieves higher accuracy with significantly lower visual token counts.
Deep Insight: Why Scaling Interaction Matters
The ablation studies provided a crucial insight: Turn budgets matter. When the researchers increased the maximum tool-use turns from 2 to 4, the performance didn't just improve—it reached a new ceiling in the later stages of training. This suggests that "thinking time" in multimodal domains behaves similarly to the "test-time compute" scaling seen in LLMs like OpenAI's o1.
However, this only works if the RL process is "filtered." As shown in the paper's analysis, without Standard Deviation Sorting, the gradient signals from correct but concise answers can actually suppress useful agentic behaviors. By keeping the variance high, the model is pushed to explore complex, multi-turn solutions that are actually robust.
Conclusion & Future Outlook
PyVision-RL proves that the "Agentic" path is the future of Multimodal AI. By moving away from passive frame sampling and toward active, Python-driven exploration, we can build models that are both smarter and cheaper to run.
The project highlights a shift in MLLM research: it's no longer just about seeing more pixels; it's about knowing which pixels to look at and how to manipulate them to find the truth.
Note: The authors have released the code and models on GitHub.