How does mutual information scale in high-dimensional neural populations?

A Monte Carlo estimator tracks mutual information in up to 10,000 neurons, revealing exponential decay of stimulus uncertainty.

Direct answer

Using a Monte Carlo estimator built on a conditionally independent population model, the study estimates mutual information in populations of roughly 100 to 10,000 neurons from visual cortex, hippocampus, and retina. Across all four datasets, stimulus uncertainty decays exponentially with population size, but the rate differs sharply by system: visual cortex approaches the stimulus entropy within a few hundred neurons, while hippocampus grows far more slowly.

1sources cited

This article was generated with WisPaper-powered search and paper analysis.

Why response patterns explode with population size

Mutual information measures how much a neural response reduces uncertainty about which stimulus was presented. Estimating it directly requires the probability of every possible population response under every stimulus. With binary neurons that is K times 2^N stimulus-response pairs for N neurons and K stimuli, so direct estimation is only feasible for populations of roughly ten cells. The paper frames this as the central obstacle: modern recordings reach thousands of neurons, but the space of response patterns grows exponentially faster.

The authors separate two difficulties that both come from this explosion. One is estimating the response distribution P(x|s) for each stimulus. The other is averaging the log-ratio log[P(x|s)/P(x)] over all stimuli and responses. Their argument is that only the first fundamentally limits reliable estimation, because the second can be handled by sampling.

Conditional independence turns the problem tractable

The paper assumes conditional independence: given a stimulus, each neuron's response is treated as independent of the others. This reduces the intractable task of estimating probabilities for all K times 2^N pairs to estimating N times K single-neuron firing probabilities P(x_i|s), which can be measured from data. The model is presented as the natural first-order description of P(x|s), not as a claim that real populations are truly independent.

To compute the information, the estimator samples a stimulus from its marginal distribution, samples a response from the conditionally independent model, and averages the log-ratio over T_s such samples. In simulations with firing probabilities drawn uniformly from 0 to 1, the estimate converged quickly, with fluctuations shrinking as 1/sqrt(T_s). The authors report the same convergence in the empirical recordings.

Finite-data bias is corrected by extrapolation

The model's firing probabilities must themselves be estimated from a finite number of experimental trials, and the paper shows this bias scales as K(N+1)/(2T_f), where T_f is the number of measured stimulus presentations. To remove it, the authors subsample different fractions f of the data, estimate mutual information for each subsample with 10^4 Monte Carlo samples, and fit a line against the inverse data fraction 1/f. Extrapolating to 1/f = 0 gives an infinite-data estimate, and repeating the procedure yields a mean and standard deviation. The paper reports that these corrections were small in all settings, which supports the reliability of the estimates.

The estimator recovers known answers in a solvable model

Before applying the method to real data, the authors test it on a model with an exact solution. Each neuron's firing probability for each stimulus is drawn from a symmetric beta distribution with shape parameter beta. Small beta makes firing nearly deterministic, while large beta pushes firing probabilities toward 1/2 so neurons decouple from the stimulus. The model therefore has a known ground-truth mutual information that can be computed analytically.

Simulating 10^5 stimulus presentations for K = 32 stimuli, the Monte Carlo estimates closely tracked the analytic values across population sizes and across beta values, including large N and strong coding at beta much less than 1. This is the paper's accuracy check: the method is validated against a case where the right answer is known, not only against itself.

Visual cortex saturates within a few hundred neurons

In mouse primary visual cortex, the authors analyze 8,991 neurons responding to 32 orientation gratings and 11,523 neurons responding to 32 natural images, both recorded with two-photon calcium imaging. Responses were binarized as active or silent. For the orientation data, the conditionally independent model quantitatively predicted correlations between neurons; for natural images it captured less of that correlation structure.

The information grew rapidly with population size in both recordings. With orientation gratings, 63 neurons were enough to halve the uncertainty about the stimulus, measured as conditional entropy H[s|x] = H[s] - I[x;s], and 613 neurons brought mutual information to within 1% of the stimulus entropy. With natural images, the corresponding numbers were 71 neurons to halve uncertainty and 513 neurons to reach within 1% of stimulus entropy. These thresholds are specific to these recordings, stimuli, and the conditional-independence assumption.

Hippocampus and retina scale more slowly

The hippocampus analysis uses 1,485 CA1 neurons recorded with calcium imaging while a mouse ran on a circular track, with position discretized into 32 equal-probability bins so the stimulus count matches the visual experiments. Here the conditionally independent model, defined by each cell's place field, captured some but not all correlations. Information grew more slowly: on average 174 neurons were needed to halve stimulus uncertainty, and even the full population did not bring mutual information to within 1% of the stimulus entropy. Varying the spatial discretization did not qualitatively change this result.

The retina analysis uses only 53 salamander neurons responding to an artificial moving bar, coarse-grained into 32 equal-probability position bins. The conditionally independent model did not capture the strong correlations between these neurons, and the encoded information was lower than in mouse cortex and hippocampus. Because this population is much smaller than the others, its curve should be read as a short segment rather than a full scaling comparison.

One exponential rate separates the systems

Across all four datasets and the simulated neurons, conditional entropy decayed exponentially with N. The paper connects this to prior theory predicting such decay for conditionally independent neurons, with an exponent equal to the Chernoff distance between the response distributions induced by different stimuli. A larger exponent means fewer neurons are needed for the same reduction in stimulus uncertainty.

The appendix derives the asymptotic rate explicitly. For independent firing probabilities across stimuli, the exponent is minus the log of the quantity formed from the mean of the square root of the firing probability plus the mean of the square root of one minus that probability. When a neuron's firing probabilities are correlated across stimuli, the slowest-decaying pair of stimuli sets the rate. This gives the paper its unifying quantity: the differences between visual cortex, hippocampus, and retina are differences in a single decay exponent, not in whether the exponential law holds.

What the scaling law does and does not settle

The practical value of this work is a way to compare how efficiently different populations and stimulus sets reduce uncertainty as neurons are added, using a measure that scales to thousands of cells. The most decision-relevant contrast is between the visual cortex recordings, which approach the 5-bit stimulus entropy within a few hundred neurons, and the hippocampal recording, which does not reach within 1% even with all 1,485 neurons. A reader comparing systems should therefore look at the decay exponent and the neuron count needed to halve uncertainty, not at raw mutual information alone.

The main boundary is the conditional-independence assumption. The paper shows where that model fits the measured correlations well, as with orientation gratings, and where it fits poorly, as with the retina and, to a lesser degree, natural images and hippocampus. In the latter cases the reported information is what the conditionally independent model encodes, and the authors note that the model captures only part of the correlation structure. The exponential scaling result is thus established within that modeling choice, and the between-system differences in decay rate are the paper's most transferable finding.

Sources used in this answer

1

Mutual Information in High-Dimensional Neural Coding

A Monte Carlo estimator under conditional independence recovers analytic ground truth, corrects finite-data bias, and scales to 100 to 10,000 neurons, showing stimulus uncertainty decays exponentially with population size across visual cortex, hippocampus, and retina.