Why key-only page summaries miss output-important pages
Page sparse attention groups consecutive KV entries into fixed-size pages, keeps a compact summary per page, and retrieves only a subset of pages for each query before running exact attention over their tokens. The paper's framing is that the pipeline's effectiveness depends on what the page summary preserves, and existing retrieval methods answer that question mainly through keys: extrema, low-dimensional key representations, geometric statistics, or spectral summaries.
The paper argues this is a mismatch. Keys determine attention logits, but what propagates to the rest of the network is the attention output, o(q) = sum over i of alpha_i(q) times v_i, where alpha(q) = softmax(Kq / sqrt(d)). The effect of an error in the attention distribution therefore depends on the associated values. A large logit error may barely change the output when attention is redistributed among tokens with similar values, while a small error along a weak key direction may substantially change the output if it shifts probability between very different values. A criterion based only on key geometry cannot distinguish these cases.
The paper's Figure 1 illustrates the consequence: a page that is important to the output can fall below the top-2 retrieval cutoff under key-only summaries and key approximations, and is recovered only when the page representation is built from keys and values together.
Two objectives have different local sensitivities
The paper separates two things a page summary must support. To contribute to the output, a page must first be retrieved, which requires an accurate estimate of its attention mass. Once retrieved, its contribution depends on the conditional attention output computed from its tokens.
The paper reports that these two objectives have different local sensitivities: conditional attention output error is generically first order in discarded logit directions, while page log mass error is locally second order. That asymmetry is the paper's stated motivation for a hybrid objective that combines output sensitivity with the conventional key covariance.
The resulting basis comes from a small page-local eigenproblem. At one endpoint it reduces to ordinary page-local key PCA; moving away from that endpoint introduces value information according to its effect on the attention output. The paper names the method OVAL and describes it as an output-aware rank-r page basis governed by centered key-value coupling, combined with key fidelity through a single normalized parameter eta, giving a closed-form spectral construction.
Values are used once, at build time
OVAL uses both keys and values when constructing each page basis, but stores no value statistics and requires no value-dependent computation during retrieval. Values are accessed only when a completed page summary is constructed. At decode time, the summary has the same dimensionality and scoring complexity as a key-only spectral page summary.
This matters for how the method should be compared. The paper's claim is not that OVAL is cheaper than key-only spectral retrieval at decode time, but that it changes the basis objective at no additional stored size or decode-time scoring cost. The paper also states the method is training-free and requires no additional value-dependent statistics at inference time.
The evaluated configuration uses page size 32, rank r = 8, and eta in {0, 0.25, 0.5, 0.75, 1}. Because centering removes one degree of freedom, a page of size 32 supports rank at most 31. Summaries are constructed in float32 and cast back to the cache dtype.
Eta trades key fidelity against output sensitivity
The single parameter eta controls how far the basis moves from the key-PCA endpoint. The paper's Table 4 reports the effect of eta across long reasoning, LongBench v2, and LongGenBench, and the pattern is not monotone: the best eta differs by task and by model.
On Llama 8B long reasoning, MATH500 peaks at eta = 0.25 (69.75), AIME24 at eta = 0.5 (53.33), and GPQA at eta = 0.75 (44.00). On Qwen 7B, MATH500 peaks at eta = 0.25 (73.75), while AIME24 and GPQA both peak at eta = 0.5 (60.00 and 36.25). On Qwen 14B, MATH500 peaks at eta = 0.5 (72.00), AIME24 at eta = 0.25 (71.25), and GPQA at eta = 0.5 (56.00).
The eta = 0 row is the key-only spectral endpoint. The paper's headline claim is that the output-aware component consistently improves over this endpoint on long reasoning tasks, and that with the best evaluated eta OVAL achieves the highest avg@k among compressed methods in all long reasoning settings, by +3.7 to +3.9 points.
Calibrated eta transfers unevenly across models
Because the best eta is task-dependent, the paper studies practical selection using a disjoint calibration set: a single eta-hat is selected per model on a calibration pool and then frozen across tasks, and all methods are scored on the same held-out examples. Table 3 reports the gap between this frozen choice and the per-setting sweep best.
For Llama 8B with eta-hat = 0.50, the frozen average is 53.69 versus a sweep best of 55.75, a gap of 2.05. The per-task gaps are 3.74 on MATH500, 0.00 on AIME24, and 2.42 on GPQA. For Qwen 7B with eta-hat = 0.25, the frozen average is 54.28 versus a sweep best of 57.78, a gap of 3.50, with per-task gaps of 1.75, 6.67, and 2.08.
The paper's own reading is that eta can be selected from held-out calibration data, that this transfers well on Llama 8B, and that substantial headroom remains on Qwen 7B. AIME24 contains no calibration problems and is therefore identical to the evaluation set in Table 1, which is why its gap is zero for Llama 8B.
Long-context understanding and generation results
On LongBench v2, the paper reports that OVAL achieves the best Overall score on two of three models. For Llama 3.1 8B Instruct, OVAL's Overall is 29.82 against FullKV 29.22, Quest 28.43, ArkVale 28.63, RaaS 28.23, InfiniGen 28.56, and FreeKV 29.22. For Qwen 2.5 7B Instruct, OVAL's Overall is 28.03 against FullKV 27.44 and Quest 27.63. For Qwen 2.5 14B Instruct, OVAL's Overall is 33.00, below FullKV 33.40, Razor 34.19, ArkVale 34.19, and ShadowKV 34.79.
On LongGenBench, OVAL's CR x Acc is 28.50 for Llama 8B, 32.38 for Qwen 7B, and 28.64 for Qwen 14B. The corresponding FullKV values are 26.82, 31.09, and 29.35. The paper describes OVAL as matching or surpassing leading baselines on several long-context understanding and generation settings with modest decoding overhead, and as performing competitively with recent KV cache compression and retrieval methods.
These are aggregate benchmark scores under a matched effective KV budget of B = 2048, not per-token or per-layer measurements. The paper notes that the accuracy and performance harnesses express this budget through different combinations of selectable, sink, and recent regions, so differences in raw CLI fields do not imply different effective budgets.
Accuracy-latency tradeoff on InfiniteBench
The paper adds an InfiniteBench evaluation to test a longer context regime than the 64K LongBench v2 setup. Table 6 reports LongQA results averaging En.MC, En.QA, Zh.QA, En.Sum, and En.Dia with 50 examples per task.
OVAL reaches 37.4 LongQA accuracy, compared with 36.0 for FreeKV, 35.9 for ArkVale, and 35.5 for RaaS, recovering 87.2% of FullKV accuracy versus 83.9% for FreeKV. This gain costs 1.7 ms per token, a 7.4% decode overhead relative to FreeKV. OVAL is also faster than both RaaS (27.4 ms) and ArkVale (76.9 ms), placing it on the accuracy-latency Pareto frontier among the evaluated retrieval methods.
A gap of 5.5 points to FullKV remains at this budget, which the paper reads as evidence that contexts beyond 100K tokens remain challenging for all compressed methods. That is a statement about this evaluated budget and benchmark, not a claim that OVAL closes the long-context gap in general.
What to take from the output-aware basis
The paper's central priority is a change of objective, not a change of format: keep the compact spectral page summary and its decode-time scoring cost, but choose the basis directions by their effect on the value-weighted attention output instead of by key energy alone. The evidence for that priority is the consistent improvement over the eta = 0 key-PCA endpoint on long reasoning, and the competitive or leading aggregate scores on LongBench v2 and LongGenBench at a matched budget of B = 2048.
The most decision-relevant comparison is the eta sweep. Because the best eta varies by task and model, and because a single calibrated eta leaves a 2.05-point average gap on Llama 8B and a 3.50-point average gap on Qwen 7B relative to the per-setting sweep best, a reader evaluating OVAL should treat eta selection as a first-class choice rather than a fixed default. The InfiniteBench result adds a second threshold: at 1.7 ms per token of decode overhead, OVAL buys 1.4 points of LongQA accuracy over FreeKV while remaining faster than RaaS and ArkVale, but still sits 5.5 points below FullKV.
Sources used in this answer
OVAL: Output-Aware Local Page Bases for KV Cache Retrieval
OVAL builds each KV page's compact retrieval basis from the joint structure of keys and values, so the preserved directions reflect the value-weighted attention output rather than key energy alone. It is training-free, stores no value statistics, and keeps the same summary size and decode-time scoring cost as a key-only spectral page summary.