[CVPR 2026] 2D or 3D: Who Governs Salience in VLA Models?

TL;DR

As embodied AI moves toward multi-modal sensing, the computation cost of processing both 2D images and 3D point clouds is skyrocketing. This paper presents a Tri-Stage Token Pruning Framework that intelligently "throws away" redundant visual tokens by understanding which modality (2D or 3D) is more important for a specific robot action at a specific time. It reaches 2.55× speedup while maintaining nearly the same success rate as heavy, unpruned models.

Background: The Price of Dimensionality

The evolution of Vision-Language-Action (VLA) models has hit a crossroad. To perform complex spatial tasks like "Closing a Laptop," 2D images aren't enough—models need 3D point clouds. However, adding 3D data doubles or triples the number of tokens, dropping the control frequency of robots from a smooth 20Hz to a sluggish 3-5Hz.

Current pruning techniques treat all visual tokens as equals. The authors of this paper argue this is a mistake: Modality Salience is the missing piece of the puzzle.

The Core Insight: Tri-Stage Modality Awareness

The researchers discovered that 2D and 3D data don't contribute equally throughout a task. They broke the MVLA pipeline into three stages to optimize them:

1. Data Preprocessing: The Feature Norm Reveal

By analyzing the $L1$ norm of hidden features, the authors found that 2D tokens are generally more "salient" than 3D tokens in early stages. Specifically, texture-rich regions rely on 2D, while geometric structures rely on 3D. They use a Dual-Threshold Mechanism to decide whether a patch should keep its 2D version, 3D version, or both.

2. Semantic Synthesis: Clustering the Workspace

Not all parts of a scene are equal. Using K-Means clustering on attention scores, the framework labels patches as:

• Background (Sbg): Mostly pruned (90% reduction).
• Robot Arm (Srob): Preserved based on geometric necessity.
• Target Object (Sobj): Strictly protected to ensure manipulation accuracy.

3. Action Iteration: Temporal Smoothness

Robots move in a continuum. Pruning decisions shouldn't "flicker" between frames. By using an Exponential Moving Average (EMA), the model predicts the salience of the next frame based on history, ensuring the robot doesn't suddenly "go blind" to a modality mid-action.

Experimental Validation

The framework was tested on the MLA (Multi-sensory Language-Action) model across various RLBench tasks and real-world robot arms.

• Efficiency: While naive pruning causes success rates to crater (e.g., from 55% down to 16.7%), the Tri-Stage framework holds the line at 47.5% - 70.0% success while running twice as fast.
• Real-World Robustness: On a Songling Piper robotic arm, the system maintained a high success rate even with noisy real-world sensors, proving the EMA mechanism's value.

Critical Analysis: Why This Matters

The genius of this work lies in the Modality Intersection Fusion. By intersecting semantic requirements with local feature representation, the model ensures it only spends FLOPs on what is "locally useful" and "globally necessary."

Limitations: The method relies on a "Cold Start" phase where it doesn't prune the first few frames to build a history. While effective, this could be optimized for ultra-fast, single-shot tasks.

Conclusion

This paper sets a new standard for efficient Embodied AI. It proves that we don't need fewer sensors; we need smarter filters. By recognizing that 2D governs texture while 3D governs structure, the Tri-Stage framework bridges the gap between high-fidelity sensing and real-time robot control.

Key Metrics Summary

• Speedup: 2.55×
• Accuracy Loss: < 3%
• Framework Overhead: 5.8%
• Modality: 2D RGB + 3D Point Cloud