A three-axis subspace that holds object coordinates
S-Space is defined for a given model and layer as a three-dimensional linear subspace of the hidden representations of object-name tokens. Its three directions correspond to a horizontal axis, a vertical axis, and a distance axis. Because the subspace is linear, an object's location can be recovered by projecting its token representation onto those three directions, producing a coordinate triple rather than a verbal label.
The authors search for this subspace in middle layers. Their stated reason is that early layers stay relatively anchored to input representations, while later layers become increasingly aligned with producing the next verbal token. The subspace is specific to a model and a layer but shared across all token positions within that layer.
Two framing choices matter for interpreting every later result. Coordinates are viewer-centered, and the horizontal and vertical axes use image-plane coordinates rather than world coordinates, without explicitly separating depth-dependent perspective effects. The authors state that these projected coordinates suffice for most of the spatial questions they consider.
Axes found by differentiating answer scores
The three directions are not hand-designed. For each axis, the authors build relative spatial questions that compare a query object with a reference object, such as asking whether an apple is left or right of a cup. They define an answer contrast as a logit difference: right minus left for the horizontal axis, above minus below for the vertical, and close minus far for the distance axis.
They then take the gradient of that contrast with respect to the object-token activations. This gradient points in the direction where perturbing the representation would most increase the corresponding answer contrast. To isolate the spatial signal, they form a role-antisymmetric difference between the query and reference gradients and average it across prompt templates, original and swapped object orders, and examples. The averaged, normalized vectors span S-Space.
The intuition given is that relative judgments treat the two objects oppositely: a change favoring right for the query should favor left for the reference, so subtracting the two gradients reinforces the shared spatial signal. Averaging across templates and orderings is meant to cancel effects from object identity, phrasing, token roles, or scene context.
Once the axes are fixed, reading out a coordinate needs only a forward pass. For a relative question, the model compares the query and reference coordinates along the queried axis; the sign of that margin selects right, above, or close when positive and left, below, or far when negative. This readout does not require the model to verbalize an answer.
The subspace stays readable off the spatial task
A natural objection is that the axes were found using spatial questions, so they might only reflect those prompts. The authors address this by reporting that S-Space is intrinsic rather than induced by specific data or prompts, and that it remains readable under non-spatial questions. The same fixed directions are said to apply to object-token representations elicited by other prompts and tasks, not only the spatial question-answering prompts used for identification.
The paper also reports that the identified orientations generalize across datasets, tasks, and prompts. Identification used constructed relative spatial questions from COCO, and readouts were evaluated on CV-Bench, EmbSpatial, and SpatialTunnel across three models: Molmo2-ER, Qwen3.5-4B, and Qwen3.6-27B.
This is the claim that turns a probe into a proposed mechanism: if the same three directions can be read from representations the model builds for unrelated questions, the subspace looks more like a persistent internal structure than an artifact of one task format.
Encoding, transforming, and selectively using locations
The paper assigns S-Space three functional roles and supports each with a different kind of evidence. Perception encodes object locations into the subspace, forming representations that can be read out to support accurate spatial judgments. Internal reasoning transforms the subspace when the viewpoint changes. Verbal responses access it selectively.
The selectivity evidence comes from causal interventions: altering S-Space changes spatial judgments while largely preserving non-spatial responses. That pattern is what distinguishes a workspace the model consults for spatial answers from a representation that merely correlates with them.
Case studies extend the picture in two directions. S-Space assigns locations to unseen entities and to the observer, and it connects physical directions with abstract linguistic meanings. The authors read this as evidence of both the capabilities and the limitations inherited from large-scale language training.
Where spatial reasoning actually breaks
The most consequential use of S-Space in this paper is diagnostic. Reading out direction words reveals a systematic coupling between cardinal directions and image-relative reference frames. In the reported failure cases, image-relative relations are incorrectly reused as cardinal directions, and the authors hypothesize that this coupling may partly reflect biases in web-scale pretraining data, where cardinal directions frequently appear in map-like frames aligned with horizontal and vertical axes. That hypothesis is offered as an interpretation, not a demonstrated cause.
The sharper result comes from decomposing performance into perception and computation. The authors report that models can form capable spatial representations yet fall short in the spatial transformations required to reason reliably across viewpoint changes. In other words, the bottleneck is not always reading locations into the workspace; it is operating on them when the observer's viewpoint moves.
The viewpoint experiment makes this concrete. On SpinBench, the authors compare direct answering, chain-of-thought reasoning, and a condition that applies the requested viewpoint rotation to the readout coordinates. With 146 questions without a spatial premise and 144 with one, the rotation condition reaches 91.10 to 99.31 across the five models, while direct answering stays between 56.25 and 72.92. Chain-of-thought sits in between, from 62.33 to 91.67 depending on model and premise condition.
These numbers should be read as a comparison of conditions on one benchmark, not as a claim that models spontaneously perform such rotations. The rotation is applied to the readout by the experimenters, which is precisely why the gap is informative: the representational content is usable, but the model's own computation does not reliably supply the transformation.
Why the diagnosis matters more than the probe
The paper's own conclusion is that mechanistic analysis of this kind shows which capabilities do and do not emerge through scaling, and for capabilities that remain unreliable, identifies the missing ingredient. Here the missing ingredient is named specifically: viewpoint transformation, not spatial encoding.
That reframes how spatial failures should be read. A wrong answer about a rotated scene is not necessarily evidence that the model failed to perceive or store where objects are. Under this account, the representation may already contain the needed coordinates, and the failure sits in the operation applied to them.
The authors draw an architectural implication as well: future systems need not rely on a monolithic model to perform every spatial operation internally. They suggest agentic spatial reasoning that connects dedicated tools for transformation, simulation, and memory, turning capable perception into reliable reasoning and action. This is a proposal grounded in the diagnosis, not a tested system in the paper.
What the evidence covers and what it does not
The empirical base is three models identified on COCO-derived relative spatial questions and evaluated on three spatial benchmarks, plus a five-model comparison on SpinBench. The subspace is model- and layer-specific, so the identified axes are not a universal coordinate system shared across architectures.
The coordinate frame is also bounded. Horizontal and vertical axes are image-plane coordinates, and depth-dependent perspective effects are not explicitly disentangled. The authors state this suffices for most questions they consider, which leaves open how far the readout transfers to tasks where true world coordinates matter.
The mechanistic claim rests on linearity: the analysis assumes spatial coordinates are linearly encoded in object-token representations, following the broader hypothesis that concepts are represented linearly in language models. If spatial information were partly encoded nonlinearly, a linear projection would capture only part of it.
Finally, the causal evidence is about altering spatial judgments while largely preserving non-spatial ones. That establishes selective access, but it does not by itself specify the circuit that performs the viewpoint transformation the models lack.
Read spatial errors as transformation failures first
The paper's most useful priority is a diagnostic ordering. When a multimodal model answers a spatial question wrongly, the evidence here suggests checking first whether the required operation is a viewpoint transformation rather than assuming the location was never represented. The perception-computation decomposition, and the SpinBench gap between direct answering and externally applied rotation, both point to transformation as the binding constraint.
The second priority is to treat reference-frame coupling as a separate failure mode. Cardinal and image-relative frames are systematically coupled in the direction-word readouts, which means errors can arise from frame confusion even when the underlying coordinates are sound. Distinguishing these two sources, transformation failure and frame confusion, is the practical contribution this paper offers beyond its probe.
Sources used in this answer
S-Space: Exploring Spatial Workspace in Multimodal Language Models
The paper identifies S-Space, a three-dimensional linear subspace in middle layers of multimodal language models where object-token representations encode viewer-centered horizontal, vertical, and distance coordinates; readouts support spatial judgments, causal interventions alter spatial but largely not non-spatial responses, and a perception-computation decomposition locates the main bottleneck in viewpoint transformations rather than spatial encoding.