What is the optimal rate for linear functional estimation in proportional high dimensions?

In dense proportional regimes with p/n>1, the paper derives a minimax rate n^-α(κ,γ) below n^-1 and a contour-integral estimator that attains it.

Direct answer

When the number of covariates exceeds the sample size (γ = p/n > 1) and neither the regression vector nor the precision matrix is sparse, a single linear functional of the coefficients can still be estimated consistently, but slower than the usual n^-1 rate. The paper shows the minimax mean squared rate is of order n^-α(κ,γ), where α(κ,γ) is given by an explicit elliptic-integral expression and is strictly less than 1 whenever both κ > 1 and γ > 1, and it proposes the Bocca estimator, which reaches that rate.

1sources cited

This article was generated with WisPaper-powered search and paper analysis.

A single coefficient is not the whole vector

The paper studies the random-design linear model Y = Xβ + ε with unknown design covariance Σ = Cov(X) and unknown noise variance σ² = Var(ε). The object of interest is not β itself but the scalar functional ⟨v, β⟩ for a known deterministic unit vector v; most of the time v is a canonical basis vector e_j, so the target is one coefficient β_j. This matters because in the proportional regime γ = p/n ∈ (1, ∞) the two problems behave differently: recovering the full vector and recovering one low-dimensional functional are not the same task.

The paper states that even under isotropic Gaussian design, the minimax ℓ2-norm rate over a fixed-radius ball need not vanish under proportional asymptotics, so full-vector recovery can fail while a fixed coordinate remains estimable. The obstacle is concrete: when p > n the sample covariance is singular, and estimating the inverse-covariance direction associated with the target adds a high-dimensional nuisance parameter. The paper's question is therefore the minimax rate of a fixed linear functional in this dense proportional regime, where both β and the precision matrix Σ⁻¹ may be dense.

Dense regime removes the sparsity shortcut

The setting is called dense because neither the regression vector nor the precision matrix is assumed sparse. That removes the route taken by debiased Lasso and related methods, which use an approximately orthogonal score or a constrained approximate inverse built on sparsity of β and Σ⁻¹. The paper notes that when the design covariance is known, method-of-moments estimation can work, but the unknown-covariance versions of those arguments need a sparse precision matrix.

Instead, the paper assumes only that the spectrum of the design covariance lies in an interval [ℓ, L]. The condition number κ enters the answer through this spectral bound, and the paper's later comparison indicates the optimal rate is less sensitive to growth in the proportion γ than to the covariance condition number κ. Readers should keep the two ratios separate: γ = p/n measures dimension relative to sample size, while κ measures how spread out the design covariance eigenvalues are.

The optimal exponent falls below one

Under the spectral assumption, the paper establishes that the minimax mean squared rate for the linear functional has order n^-α(κ,γ), where the exponent α(κ,γ) has an explicit elliptic-integral representation. The key qualitative result is that α(κ,γ) < 1 when both κ > 1 and γ > 1. In other words, the achievable mean squared error decays more slowly than n^-1 in this dense proportional regime, and the shortfall is governed by the two quantities that define the regime.

This is a statement about a rate, not about a particular finite-sample constant. The exponent is a function of κ and γ, so the same sample size n can correspond to different decay behavior depending on the design's condition number and on how far p/n sits above one. The paper also reports a bound of order 1/(4√κ log γ) in the vicinity of its rate analysis, which it reads as evidence that the optimal rate is less sensitive to growth in γ than to κ.

Bocca combines calibrated resolvents on a contour

The proposed Bocca estimator is built from empirically calibrated resolvents combined through a complex contour integral. A resolvent here means the matrix inverse (S − zI)⁻¹ evaluated at a complex argument z, and the estimator integrates such quantities along a contour rather than inverting the sample covariance directly. This is how the method sidesteps the singularity of the sample covariance when p > n: the contour is chosen away from the problematic real spectrum.

The computation is explicit. The contour circle Γ_δ is discretized into M equally spaced nodes, and at each node the paper solves for a root ζ̂(μ) of the equation (1/n)Σ_i 1/(s_i − z) + 1/μ = 0, where s_i are the eigenvalues of Ŝ = XXᵀ/n. The first node is handled by bisection on a stated interval, and subsequent nodes by Newton's method using the derivative formula for ζ̂′(μ). A further property the paper highlights is that the estimator is linear in v before clipping, so it can be computed once and then evaluated for any loading vector v.

Matching the lower bound requires calibrated roots

The paper's argument has two sides. On the lower-bound side, it constructs two hypotheses whose data laws are close in total variation while the target coefficient differs by at least a constant times n^-α(κ,γ)/2, which forces any estimator to incur mean squared error of the stated order. The construction uses a mixing distribution over covariance parameters, with the determinant factor in the mixing density arranged so that the same matrices appear under both hypotheses.

On the upper-bound side, the Bocca estimator must track the deterministic resolvent closely enough that the contour integral inherits the optimal rate. The paper proves uniform control of the empirical root ζ̂(μ) relative to its deterministic counterpart over the contour, with a deviation bound of order n^-3/4 δ^-c, and shows the empirical inverse exists on an event whose complement has polynomially small probability. These are the technical steps that let the estimator attain, rather than merely approach, the minimax rate.

Scope is dense, proportional, and spectral

The conclusion is tied to a specific regime: γ = p/n in (1, ∞), unknown design covariance, no sparsity in β or Σ⁻¹, and a design spectrum confined to [ℓ, L]. The rate is a minimax mean squared rate for a single linear functional, not a full-vector recovery guarantee, and the paper explicitly contrasts the two. It also does not claim the rate holds outside the proportional asymptotics or without the spectral bound.

The paper's own framing of the contribution is that the comparison does not assume an estimable full precision matrix when that matrix is unknown. That is the practical distinction worth carrying away: the method targets one prespecified contrast, such as an exposure coefficient after adjustment for many covariates or a contrast among effects in a genomic study, in settings where assuming only a few covariates matter is not defensible.

Condition number matters more than the dimension ratio

The most decision-relevant comparison in the paper is between the two arguments of α(κ,γ). Because α(κ,γ) < 1 whenever κ > 1 and γ > 1, the dense proportional regime always pays a price relative to the n^-1 rate, but the paper's bound of order 1/(4√κ log γ) indicates that the rate degrades faster with the covariance condition number κ than with the dimension ratio γ. A reader comparing designs should therefore look first at how ill-conditioned the design covariance is, and second at how far p/n exceeds one.

The estimator's own requirements point the same way. Its computation depends on a contour placed relative to the spectrum and on root-finding that stays stable as the spectral margin shrinks, so the spectral spread is not just a rate parameter but also a computational one. The paper's contribution is thus best read as a rate result with an accompanying algorithm: the exponent n^-α(κ,γ) sets what any method can achieve in this regime, and the Bocca estimator is the construction shown to achieve it.

Sources used in this answer

1

Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models

Establishes the minimax mean squared rate n^-α(κ,γ) for a linear functional in dense proportional regimes with γ = p/n > 1 and unknown design covariance, with α(κ,γ) < 1 when κ > 1 and γ > 1, and proposes the Bocca estimator, which attains this rate via empirically calibrated resolvents combined in a complex contour integral.