[CVPR 2025] DeepScan: Elevating LVLM Reasoning through Bottom-Up Visual Grounding
DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models
DeepScan is a training-free framework for Large Vision-Language Models (LVLMs) that enhances visually grounded reasoning through a bottom-up pipeline. It integrates Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning to achieve state-of-the-art performance on benchmarks like V* (90.6% accuracy) and TreeBench.
TL;DR
DeepScan is a novel, training-free framework that enables Large Vision-Language Models (LVLMs) to "think before they speak" by explicitly localizing and recalibrating visual evidence. By replacing fragile one-shot localization with a bottom-up Hierarchical Scanning approach, it achieves a massive +16.3% boost on the V* benchmark and outperforms many RL-tuned models without any adaptation cost.
Background Positioning
In the current landscape of LVLMs, models like GPT-4o and Qwen2.5-VL have made strides in general perception but still struggle with "ultra-fine-grained" visual reasoning—such as finding a tiny object in a high-resolution 8K image. Most existing solutions use a top-down strategy (look at the whole image first, then crop). DeepScan flips this script, proposing a human-inspired bottom-up paradigm that scans local patches to find subtle cues before relating them to the global context.
The Core Challenge: Attention Drift and Sinks
Why do powerful models fail to find a small "man in a white cap" in a landscape? The authors identify two primary culprits:
- Attention Sink: The model's attention is captured by unintended, non-semantic background regions.
- Attention Drift: The model incorrectly focuses on a salient object that is semantically similar to the target but ultimately irrelevant.
Traditional coarse-to-fine methods often get "stuck" in these sinks during the first global look. DeepScan avoids this by never starting with the "whole picture" when looking for fine-grained details.
Methodology: The DeepScan Pipeline
DeepScan operates through three sophisticated stages:
1. Hierarchical Scanning (Bottom-Up Paradigm)
Instead of asking "Where is the object?", DeepScan slices the image into patches and asks "Are there any cues here?".
- Local Cue Exploration: Uses a search expert (like BLIP-ITM) to generate attention maps in small patches.
- Point-Based Proxies: Instead of messy bounding boxes, it uses specific points (Attention Peaks and Chebyshev Centers) to represent cues.
- Multi-scale Extraction: Points are passed to a visual expert (like LangSAM) to recover full evidence masks, which are then enhanced via morphological post-processing.
2. Refocusing (Context Optimization)
Finding the object is only half the battle; the model needs the right amount of context to reason about it. Too much context leads to noise; too little leads to loss of spatial relationships. Refocusing uses a Zoom-In/Zoom-Out search strategy. It treats the view selection as an optimization problem, seeking the smallest possible view that still allows the LVLM to confidently confirm the presence of all necessary targets.
3. Evidence-Enhanced Reasoning
Finally, the model builds a Hybrid Evidence Memory. It doesn't just look at the final crop; it looks at the original image, the fine-grained evidence, and the recalibrated views simultaneously. This multi-granular input allows the model to resolve attributes (e.g., color) and relations (e.g., "to the left of") with high precision.
Experimental Results: SOTA without Training
The performance gains are most striking on tasks requiring needle-in-a-haystack perception.
- V Bench*: Achieves 90.6% (integrated with Qwen2.5-VL-7B), a huge jump from the baseline's 74.3%.
- Generalizability: It works across architectures (LLaVA, InternVL, Qwen) and scales (7B to 72B).
One of the most interesting findings from the Ablation Study is that "More grounding is not always better." If you crop too tightly (high IoU), the model loses the spatial context needed for reasoning, causing performance to drop. DeepScan’s "Refocusing" stage specifically solves this by finding the "Goldilocks zone" of context.
Critical Insight: The "Bottom-Up" Advantage
As shown in the qualitative comparisons, while models like GPT-4o might hallucinate or drift toward "Attention Sinks," DeepScan's patch-wise scanning forces the model to ignore global distractions.
Takeaway: Systematically scaling test-time compute to maximize the visual Signal-to-Noise Ratio (SNR) is more effective than heuristic search-tree expansions.
Conclusion & Future Work
DeepScan proves that we don't always need more parameters or more RLHF to solve complex visual tasks. Sometimes, we just need a better way to look at the data. While the current framework has a higher latency than one-shot models, engineering optimizations like batching and vLLM integration make it a viable "Thinking Mode" for the next generation of visual agents.
Limitations: Currently, it struggles if the visual experts provide highly ambiguous evidence or if multiple targets are so widely separated that even a "Refocused" view becomes noisy. Future work involving "Generative Composition" to reassemble distant evidence into clean, compact layouts holds promise.