How much brain-LLM alignment comes from interpretable semantic properties?

Variance partitioning shows most LLaMA2-7B brain predictivity is shared with 65 interpretable semantic properties, not unique to the LLM.

Direct answer

In this study, most of LLaMA2-7B's brain-predictive variance was shared with SCRIBE, an interpretable representation of 65 semantic properties, across the whole brain and language-related regions. Both representations still kept significant unique contributions, so the alignment is largely, but not entirely, describable in interpretable semantic terms.

1sources cited

This article was generated with WisPaper-powered search and paper analysis.

Shared variance dominates, but neither representation is redundant

The paper asks what linguistic information drives the match between a large language model's internal representations and brain responses during naturalistic story listening. Its answer is that a substantial portion of that match is associated with interpretable semantic information, while part of it is not. The authors reach this by comparing LLaMA2-7B with SCRIBE, a representation that scores each content-bearing word on 65 semantic properties spanning perceptual, motor, spatial, temporal, social, cognitive, and affective dimensions.

The comparison uses variance partitioning: three ridge-regression encoding models are fit to fMRI responses, one using LLaMA2-7B features, one using SCRIBE features, and one using both concatenated. Squaring each model's held-out prediction correlation gives explained variance, and simple subtraction splits it into variance shared by both representations, variance unique to LLaMA2-7B, and variance unique to SCRIBE. Shared variance was the dominant component across the whole brain and the language-related regions examined, yet both unique components remained statistically significant.

That distinction matters because a high brain-prediction score alone cannot say whether the model is capturing something interpretable. Here the shared component is the part of LLaMA2-7B's predictivity that SCRIBE's explicit semantic dimensions also capture; the LLaMA-unique component is what remains after SCRIBE is accounted for.

Seven participants, one model layer, one story dataset

The brain data come from seven participants listening to autobiographical stories from the Moth Radio Hour while undergoing fMRI. Word-level representations were aligned to the fMRI signal and expanded with four delayed copies covering roughly 2 to 8 seconds to account for the hemodynamic response. For each voxel, an encoding model was trained on nine stories and tested on a held-out story, with analyses restricted to voxels whose estimated noise ceiling exceeded 0.4.

LLaMA2-7B features came from layer 15 of 32, an intermediate layer chosen because intermediate layers tend to be among the most brain-predictive. SCRIBE scores were produced by an ensemble of four LLM annotators and had previously been validated against contextual semantic judgments from 921 human participants, reaching 75–81% of the estimated human noise ceiling.

One asymmetry is built into the design: LLaMA2-7B saw a sliding context window of 256 to 512 preceding words, while SCRIBE used a 10-token window. The authors kept each representation in its standard configuration and note that LLaMA-unique variance may therefore partly reflect longer-range context. Because the study covers one model, one layer, one dataset, and seven participants, the conclusions are specific to this setting.

Concrete properties share more variance than abstract ones

To ask which kinds of meaning carry the shared signal, the authors used concreteness ratings on a five-point abstract–concrete scale, available for 61 of the 65 SCRIBE properties. They selected the 10 most concrete and 10 most abstract properties and fit separate SCRIBE and joint models for each set.

Concrete properties showed greater shared variance with LLaMA2-7B than abstract properties across the whole brain and most examined regions. The reverse did not hold: abstract properties did not show greater SCRIBE-unique variance. In other words, concreteness tracked how much a property's brain-predictive signal overlapped with the LLM, not how much semantic information SCRIBE added beyond the LLM.

This is a comparison between two selected sets of 10 properties each, not a claim that every concrete property behaves this way. It also concerns shared and SCRIBE-unique variance specifically, so it should not be read as a general statement about which properties the brain represents most strongly.

Semantic contributions accumulate across many properties

The second property-level analysis removed one SCRIBE dimension at a time across all 65 and retrained the SCRIBE and joint models, ranking properties by their mean whole-brain decrease in shared and SCRIBE-unique variance. Because the properties are correlated, a leave-one-out decrease measures a property's unique contribution given the other 64, not its overall importance.

From those rankings the authors built nested top-5, top-10, and top-20 sets, removed each set, and recomputed the variance partitioning. The result was cumulative: semantic contributions extended across multiple dimensions, with cumulative effects on both shared and SCRIBE-unique variance. No single property carried the alignment.

The authors also flag a measurement issue: SCRIBE-unique variance is small, so its percentage changes are unstable. They therefore ranked and tested on absolute decreases and summarized SCRIBE-unique changes by the median percentage across participants. With seven participants, the smallest attainable p-value in the paired one-sided exact Wilcoxon tests is 1/27 ≈ 0.0078, reached only when all seven show an effect in the same direction, which is why the paper also reports how many participants show each effect.

Shared variance is human-aligned semantics, with a caveat

The paper's interpretation is that, for LLaMA2-7B on this dataset, a substantial portion of brain-LLM alignment is associated with interpretable semantic information, while SCRIBE reveals complementary brain-predictive semantic information beyond the LLaMA2-7B representation. That framing moves the question from whether an LLM predicts brain responses to which interpretable information underlies the prediction.

A caveat qualifies the shared component. Two of SCRIBE's annotators, Llama-3.1-8B and Llama-3.1-70B, share LLaMA2-7B's model family. The authors argue that SCRIBE's validation against human judgments indicates the shared component reflects human-aligned semantics, but they acknowledge that an LLM-specific contribution cannot be fully excluded.

The study also leaves open how sensitive the result is to the chosen layer and whether it generalizes to newer or larger LLMs, whose brain alignment varies with scale. Those are untested boundaries rather than findings.

The result is a decomposition, not a causal claim

The variance components are statistical partitions of held-out prediction variance, not causal statements about what the brain uses. Shared variance means both representations predict the same brain-response variance; it does not establish that LLaMA2-7B computes meaning the way the brain does, nor that SCRIBE's properties are the mechanism behind the alignment.

The practical priority the paper supports is narrower and more useful: when evaluating brain-LLM alignment, the shared-versus-unique split tells you how much of a model's predictivity an interpretable semantic basis can already account for. Here that shared portion was dominant, and the concrete-versus-abstract comparison and the cumulative ablation effects describe where the remaining interpretable signal sits.

For readers comparing models or representations, the paper's own threshold is the noise-ceiling cutoff of 0.4 and its region set of whole brain, late language network, angular gyrus, lateral temporal cortex, and IFG/MFG. Conclusions drawn outside that voxel selection, region set, model, layer, or dataset are not tested here.

Sources used in this answer

1

Decomposing Brain-LLM Alignment with Interpretable Semantic Representations

Variance partitioning between LLaMA2-7B and SCRIBE on Moth Radio Hour fMRI data from seven participants shows shared variance is dominant across the whole brain and language-related regions, while both representations retain significant unique contributions; concrete properties share more variance than abstract ones, and property ablations show cumulative effects.