How can multiple regression quantities be read from an inscribed ellipsoid?

One diagram encodes three regression quantities

When explanatory variables are correlated, the numbers alone make it hard to see how coefficients, marginal correlations, and partial correlations relate. A coefficient can carry the opposite sign to the marginal correlation between its variable and the response, a standardized coefficient can exceed 1 in absolute value, and the multiple correlation can be much larger than every marginal correlation. This paper builds a single configuration in Rᵖ, determined only by the (p+1)×(p+1) correlation matrix of (X, Y), from which three kinds of regression quantities are read off by elementary geometric constructions.

The inputs are the p×p correlation matrix R of the explanatory variables and the vector r of their correlations with the response. The ellipsoid E = {x : xᵀR⁻¹x = 1} is the data ellipsoid of the standardized explanatory variables. Lemma 7 shows it is inscribed in the cube [−1,1]ᵖ and touches the face xⱼ = 1 at Tⱼ, the j-th column of R; the map from positive definite correlation matrices to such ellipsoids is a bijection. Lemma 8 places P = r on or inside E, strictly inside when the full correlation matrix of (X, Y) is positive definite. Theorems 9, 10, and 12 then read the three quantities from this one picture. The construction holds in any finite dimension p, though direct visualization is generally practical for p ≤ 3.

Multiple correlation is a ratio of two lengths

Theorem 9 states that the multiple correlation ρX⊢Y equals |OP|/|OP′|, where O is the origin and P′ is the point where the ray from O through P meets E (excluding P = O). The proof is short: since ‖cP‖R⁻¹ = c‖P‖R⁻¹ for positive c, the point P′ = P/‖P‖R⁻¹ is the unique point of the ray with ‖x‖R⁻¹ = 1, so the ratio equals ‖P‖R⁻¹, which Lemma 8(a) identifies with ρX⊢Y. In other words, the multiple correlation is the norm of r in the R⁻¹ metric, a Mahalanobis length.

Three special positions clarify what the ratio measures. If R is the identity, E is the unit sphere and ρX⊢Y reduces to the Euclidean length of P, the square root of the sum of squared marginal correlations. If P lies on the line through O and some Tᵢ, so that P = ρᵢTᵢ, then ρX⊢Y = |ρᵢ| and the other variables do not improve linear prediction of Y. The third case is enhancement, where ρ²X⊢Y exceeds the sum of squared marginal correlations; by Theorem 9 this happens exactly when |OP′| < 1, meaning E is narrower than the unit sphere in the direction of P. For p = 2 with ρ₁₂ = 0.8 and r = (0.7, 0.2)ᵀ, the squared multiple correlation is 0.85 while the squared Euclidean length of P is 0.53, giving |OP′| ≈ 0.790.

Standardized coefficients are coordinates in the column basis

Theorem 10 reads the standardized regression coefficients from the same point. Because T₁, …, Tₚ are the columns of the invertible matrix R, they form a basis of Rᵖ, and P decomposes as P = Σᵢ b̂#ᵢ Tᵢ, where b̂#ᵢ is the i-th standardized coefficient. Equivalently, each coefficient is the value at P of the linear functional fᵢ defined by fᵢ(Tⱼ) = δᵢⱼ, with the explicit form fᵢ(x) = eᵢᵀR⁻¹x.

The level sets fᵢ(x) = c are parallel hyperplanes: fᵢ(x) = 0 passes through the origin and is spanned by the other columns, while fᵢ(x) = ±1 pass through ±Tᵢ. This hyperplane is the diametral hyperplane of E conjugate to the coordinate direction eᵢ, so every chord of E parallel to the xᵢ-axis has its midpoint there. Holding R and the other correlations fixed, as P moves along that chord the standardized coefficient of Xᵢ is zero exactly at the midpoint. That midpoint is also the zero of the ruler used for the partial correlation, which links the second and third constructions.

The paper notes that standardized coefficients can fall outside [−1, 1], and its worked examples illustrate both that and sign reversal. These are properties of the standardized scale, not of the raw regression coefficients on the original measurement scales.

Partial correlation is a ruler reading on a chord

Theorem 12 fixes a variable i and assumes the chord ℓᵢ ∩ Ē has positive length, where ℓᵢ is the line through P parallel to the xᵢ-axis. Let Q±ᵢ be the chord endpoints and let gᵢ be the affine bijection from [−1, 1] to the chord with gᵢ(−1) = Q−ᵢ and gᵢ(+1) = Q+ᵢ, oriented so the i-th coordinate increases. Then gᵢ⁻¹(P) equals the partial correlation between Xᵢ and Y given the other explanatory variables.

The closed form makes the reading explicit. With vᵢ = (R⁻¹)ᵢᵢ and Dᵢ = (b̂#ᵢ)² + vᵢ(1 − ρ²X⊢Y), the chord endpoints are P + t±ᵢeᵢ with t±ᵢ = (−b̂#ᵢ ± √Dᵢ)/vᵢ, and the ruler value at P is b̂#ᵢ/√Dᵢ. The proof expands ‖P + teᵢ‖²R⁻¹ as ρ²X⊢Y + 2b̂#ᵢt + vᵢt², whose roots give the chord, and then identifies the normalized position with the partial correlation. The paper also states that the parallel chord through the origin has half-length 1/√VIFᵢ, tying the picture to the variance inflation factor.

Two boundaries matter here. The partial correlation is defined as the correlation of the residuals of Xᵢ and Y after linear regression on the remaining explanatory variables, and it requires both residual variances to be positive. The ruler construction additionally requires the chord to have positive length, which the paper records as Assumption 11.

Sign reversal and variance inflation become visible

Section 5 turns the diagram into statements about sign reversal, suppression, bounds on coefficients, and variance inflation. The introduction frames the underlying phenomena: a coefficient may have the sign opposite to the marginal correlation between its variable and Y, and then the partial correlation shares the sign of the coefficient; a standardized coefficient may exceed 1 in absolute value; and the multiple correlation may be much larger than every marginal correlation. The paper attributes the algebraic explanations of these phenomena to earlier work under the headings of suppression, multicollinearity, and enhancement, and positions its contribution as making them visible in one configuration.

The variance inflation factor enters through the chord geometry. The paper defines VIFᵢ as (R⁻¹)ᵢᵢ, equivalently 1/(1 − ρ²X₍₋ᵢ₎⊢Xᵢ), and Proposition 15 connects it to the chord through the origin parallel to the xᵢ-axis, whose half-length is 1/√VIFᵢ. The paper also notes that its central-chord half-length describes a different construction from the horizontal-width ratio √VIF used by Friendly, Monette, and Fox. This is a comparison of two geometric encodings, not a claim that one replaces the other.

Feasible correlations also appear in the picture: Remark 13 reads the admissible range of a marginal correlation ρᵢ as the chord Q−ᵢQ+ᵢ. The paper cites Kiesl and Rässler for describing feasible correlation vectors with an ellipsoid when some variables are never jointly observed.

Worked examples and numerical checks

Section 6 gives two worked examples. Example 16 uses 186 team-season observations from 31 seasons of Nippon Professional Baseball's Central League in which every team played 130 games, drawn from the 1954–2003 records. The response is final rank, and the two explanatory variables are runs scored and runs allowed. The pooled Pearson correlations are ρ₁ = cor(runs scored, rank) = −0.4180, ρ₂ = cor(runs allowed, rank) = +0.4724, and ρ₁₂ = cor(runs scored, runs allowed) = +0.4232. For this configuration the multiple correlation read from the diagram is 0.829, and the standardized coefficients are b̂#₁ = −0.752 and b̂#₂ = 0.790.

The example illustrates the phenomena the paper sets out to make visible. The marginal correlation of runs scored with rank is negative, and the standardized coefficient keeps that negative sign, while the two explanatory variables are themselves positively correlated. The paper also reports a simulation check: for p in {2, 3, 4, 6, 10} and 200 trials each, geometric readings of the multiple correlation, standardized coefficients, and partial correlations were compared against direct least-squares and inverse-correlation computations, with all.equal tolerances of 1e-8, and all checks passed.

The baseball data are observational team-season aggregates, so the correlations describe association among those 186 observations rather than a controlled effect of scoring or allowing runs. The geometric identities themselves are algebraic and hold whenever R is positive definite; the example is an illustration, not a test of a causal claim.

Where the construction applies and where it stops

The construction is valid in any finite dimension p, but the paper states that direct visualization is generally practical only for p ≤ 3. That is a practical limit on drawing the figure, not on the algebra: the theorems are proved for general p, and the simulation check covers p up to 10. Readers who want the geometric intuition should treat the two- and three-dimensional pictures as the working cases, while the formulas remain the operative tool in higher dimensions.

The setup assumes R is positive definite, so the explanatory variables are not perfectly collinear, and the full correlation matrix of (X, Y) is positive semidefinite, becoming positive definite unless Y is an exact linear combination of the explanatory variables plus a constant. When the full matrix is not positive definite, P lies on the boundary of E rather than strictly inside. The partial correlation ruler additionally requires the chord through P parallel to the xᵢ-axis to have positive length.

The paper also positions itself against two existing geometric approaches: vectors and projections in the space of observations, and data ellipsoids in the space of the variables. Its stated addition is a single configuration in Rᵖ whose inputs are only R and r, encoding correlations as coordinates of points in a fixed cube rather than as angles between vectors. The paper reports that preliminary versions of Theorems 9, 10, and 12 were announced without proofs in a conference abstract, and that the present paper supplies complete proofs, the closed form of the ruler reading, and the consequences.

Read the chord midpoint before the endpoints

The most decision-relevant feature of this diagram is not the multiple correlation ratio but the chord structure that carries both the standardized coefficient and the partial correlation. For a fixed variable i, the midpoint of the chord through P parallel to the xᵢ-axis is where the standardized coefficient of Xᵢ is zero, and the ruler reading at P gives the partial correlation. That means a reader comparing correlated explanatory variables should look at where P sits relative to the chord midpoint, not only at the marginal correlations or at the length ratio |OP|/|OP′|.

The worked example makes the priority concrete. With ρ₁ = −0.4180, ρ₂ = +0.4724, and ρ₁₂ = +0.4232, the multiple correlation is 0.829, yet the standardized coefficients are −0.752 and 0.790, and the marginal and standardized signs for runs scored agree. The chord geometry is what shows how the two correlated explanatory variables divide their contribution, and the variance inflation factor appears there as the half-length 1/√VIFᵢ of the parallel chord through the origin. For readers deciding which quantity to inspect first when explanatory variables are correlated, the paper's own ordering points to the chord and its midpoint as the place where sign reversal, coefficient bounds, and multicollinearity become visible together.

Sources used in this answer