Geo3DPruner: Slashing 90% of Visual Tokens with Geometry-Guided Precision

TL;DR

As Video-Language Models (VideoLMs) venture into 3D scene understanding, they face a massive computational bottleneck: the "token explosion" from processing multi-view spatial videos. Geo3DPruner solves this by using the underlying geometry of the scene to prune redundant visual tokens. By modeling cross-view consistency and spatial diversity, it can discard 90% of visual tokens while maintaining over 92% of original performance, significantly boosting inference efficiency for complex 3D tasks.

The Problem: The Redundancy of 2D Projections

When we represent a 3D room as a sequence of video frames, we create massive redundancy. A wooden desk or a swivel chair might appear in 10 different frames from 10 different angles.

Standard pruning methods (like FastV) look at token "saliency" within a single frame or relative to a text query. However, they miss the View Consistency: they don't realize that a token in Frame 1 and a token in Frame 15 might represent the exact same physical voxel in 3D space. This leads to redundant computation and a fragmented understanding of the scene.

Methodology: The Geometry-Guided Two-Stage Pruning

The authors propose that visual token pruning shouldn't just be about "what looks important," but "what fills the 3D space."

1. The Geometry Backbone

Geo3DPruner employs a parallel architecture. While a 2D encoder extracts visual features, a 3D Geometry Encoder (VGGT) predicts camera parameters and depth maps. This allows the model to project every 2D pixel token into a 3D world coordinate and assign it to a Voxel.

2. Stage I: Intra-Voxel View Consistency Pruning (VCP)

Within a single voxel (a small cube in 3D space), the model might have tokens from 5 different camera views. VCP evaluates the contribution of these multi-view features using an attention submatrix. It keeps only the "best" representative tokens for that specific spatial spot, effectively filtering out multiple redundant sightings of the same object.

3. Stage II: Inter-Voxel Spatial Diversity Pruning (SDP)

To avoid "object-centric bias"—where the model focuses all its attention on one flashy object (like a TV) and ignores the rest of the room—SDP uses an iterative selection process. It calculates global attention across all voxels and picks a diverse set that ensures the entire scene structure is preserved.

Results: Efficiency Without Sacrifice

The performance metrics on benchmarks like ScanRefer and ScanQA are striking.

• Extreme Compression: At a 90% pruning ratio, Geo3DPruner maintains 92% of the baseline performance.
• The "Better than Baseline" Effect: At 20% pruning, the model actually outperforms the uncompressed baseline. This suggests the pruning acts as a "denoiser," removing visual clutter and helping the LLM focus on the essential 3D structure.

Deep Insight: Why Geometry Beats Tokens

The success of Geo3DPruner highlights a fundamental shift in AI architecture. In the past, we treated everything as a "sequence of tokens." However, for spatial intelligence, the Inductive Bias of 3D geometry is a far more efficient language. By grounding visual tokens in a 3D voxel grid, Geo3DPruner converts a chaotic video stream into a structured, compact map that an LLM can actually reason about without drowning in data.

Conclusion

Geo3DPruner is a significant step toward making 3D spatial intelligence practical for real-time applications. By leveraging the physical logic of the world—that objects exist in 3D space across time—it proves that we don't need more data to understand a scene; we just need to organize the data we already have more intelligently.

Future Outlook: The next step will naturally be applying this to streaming 4D environments (moving objects) and integrating it into embodied agents where inference speed is literally a matter of safety.