How can KV cache compression make GUI agents more efficient?

The full KV cache blocks local deployment

A GUI agent repeatedly feeds high-resolution screenshots to a vision-language model and generates actions step by step. The key-value (KV) cache stores previously computed key and value states so the model does not recompute them, but with image-heavy history it grows large. The paper reports that UI-TARS-1.5-7B can consume over 80GB of GPU memory in a single inference when fed five screenshots with a maximum of 50 steps, which can trigger out-of-memory errors on most consumer GPUs. The authors frame compression not as an optional speedup but as a prerequisite for running these agents on local machines and edge devices.

The paper also gives a concrete latency reference: running UI-TARS-1.5-7B in bfloat16 with flash attention 2 through Hugging Face inference on a five-screenshot example from OSWorld-Verified can take over 15 seconds per step on an H200 on average. That number describes one model, one hardware setup, and one step, so it should not be read as a general latency figure for all GUI agents.

GUI attention sparsity stays flat across layers

The paper's first contribution is an analysis of attention sparsity, meaning the share of attention scores that fall below a relative threshold and can be treated as negligible. Using a 1% relative threshold, the authors measure sparsity across transformer layers for GUI trajectories from ScreenSpotV2, covering web, mobile, and desktop environments. They find that sparsity averages above 0.99 across all layers, with the lowest values around 0.98.

The comparison that matters is with natural images, which the paper cites as averaging 0.92 with minimums around 0.88, and which show decreasing sparsity in deeper layers. GUI screenshots instead produce an almost flat line across depth. Because the layer-to-layer differences are so small, methods that normalize sparsity into per-layer budgets can amplify tiny numerical differences into large budget disparities. The authors argue this is why layer-varying schedules designed for natural images misallocate cache on GUI tasks.

A uniform budget outperforms layer-varying schedules

To test that hypothesis, the paper compares three allocation strategies on ScreenSpotV2: an equal budget across all layers, PyramidKV's heuristic that gives more budget to shallower layers, and VL-Cache's computed budgets based on normalized attention sparsity. The uniform strategy performs significantly better than both alternatives across compression ratios. The gap widens at higher compression, particularly when only 5% to 20% of the original cache is retained, which the authors interpret as evidence that misallocated budgets become more damaging as cache resources shrink.

This is a comparison of allocation policies under the same compression setting, not a claim that uniform allocation is optimal for every multimodal workload. The paper's own framing is that prior allocation strategies developed for natural images and text do not transfer to GUI agents, and that the flat sparsity profile makes uniform allocation the appropriate default here.

Spatial saliency keeps semantically important visual tokens

GUI-KV is a plug-and-play compression method that requires no retraining. Its first component, spatial saliency guidance, augments attention scores with the L2 norm of hidden states. The purpose is to preserve visual tokens that are semantically important even when their raw attention score is not the highest. The paper motivates this with prior observations that certain tokens act as information sinks, and it reports that this residual-stream signal helps surface important visual tokens.

The second component, temporal redundancy scoring, targets the fact that consecutive screenshots share static backgrounds and persistent UI elements. At each step, the method computes a low-rank QR decomposition of the current frame's key matrix to define a subspace, then projects previous frames' keys onto that subspace. Tokens whose information already lies in the current frame's span get small residual scores and are pruned preferentially, while tokens capturing localized UI changes keep large residuals and are retained. Text tokens and current-frame visual tokens bypass this temporal filter entirely, so explicit instructions and the immediate view are never penalized by inter-frame redundancy checks.

Near full-cache accuracy at 10–20% budgets

The evaluation covers six benchmarks spanning visual grounding, offline action prediction, and online evaluation, using UI-TARS-1.5-7B and OpenCUA-7B. Across these settings, GUI-KV outperforms competitive KV compression baselines and closely matches full-cache accuracy at modest budgets. On UI-TARS-1.5-7B, retaining 10–20% of the cache typically closes most of the performance gap on visual grounding, and competitive step accuracy is maintained on action-prediction tasks. The paper also notes that at intermediate budgets the method occasionally exceeds full-cache accuracy.

The headline efficiency result is specific: in a five-screenshot setting on AgentNetBench, GUI-KV reduces decoding FLOPs by 38.9% while increasing step accuracy by 4.1% over the full-cache baseline. These two numbers come from the same comparison and should be read together. The FLOP reduction measures decoding compute per generated token, while the accuracy change measures step-level task performance; neither substitutes for the other. The paper reports prefill overhead as an increase below 0.29%, and states that savings grow with more screenshots.

Spatial and temporal gains depend on screenshot count

The ablations separate the two mechanisms. Spatial saliency is more effective when the number of screenshots is small, while temporal redundancy becomes increasingly beneficial as more screenshots are provided. Combining both yields the best performance across settings. This pattern follows from what each component does: with few frames there is little history to deduplicate, so preserving salient current-frame tokens matters more; with many frames, the overlap between consecutive screenshots grows and pruning redundant history pays off.

That split is the paper's most practical guidance for choosing where to invest effort. It also sets the boundary of the result: the reported gains are measured on GUI agent benchmarks with these two model backbones, and the paper does not claim the same balance for other multimodal tasks or other agent architectures.

Compression gains are bounded by the tested setting

The evidence supports a specific claim: exploiting GUI-specific spatial and temporal redundancy lets a training-free KV cache method retain 10–20% of the cache while staying close to full-cache accuracy on the tested benchmarks. The paper's own discussion ties the practical payoff to reduced memory footprint and bandwidth, which it says enables deployment on standard consumer hardware. That is an inference about deployment conditions, not a measured hardware result in the reported experiments.

The scope conditions are equally specific. The analysis of flat sparsity comes from ScreenSpotV2 trajectories with UI-TARS-1.5-7B, and the efficiency numbers come from a five-screenshot AgentNetBench setting. The method assumes GUI screenshots with large uniform regions and strong inter-frame overlap; the paper does not establish how it behaves when those assumptions weaken. Readers comparing compression methods should therefore check the budget, screenshot count, and benchmark before treating the 38.9% FLOP reduction as transferable.

Start with a uniform budget, then add temporal pruning

The paper's most decision-relevant finding is an ordering of priorities. Because GUI attention sparsity is uniformly high across layers, the first move is to stop varying budgets by layer and allocate uniformly; the ScreenSpotV2 comparison shows uniform allocation beating both PyramidKV and VL-Cache, with the gap widening at 5–20% retention. The second move is to add temporal redundancy scoring once multiple screenshots are in play, since its benefit grows with screenshot count while spatial saliency carries more of the load when frames are few.

That ordering, rather than a single compression ratio, is what the evidence supports. A reader deciding whether to adopt GUI-KV should look first at how many screenshots their agent keeps in context and at the budget they can afford, because those two variables determine which component matters and how close to full-cache accuracy the result is likely to land.

Sources used in this answer