Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning

[arXiv 2025] Truth as a Trajectory: Is LLM Reasoning a Moveable Feast?

Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning

Summary

Problem

Method

Results

Takeaways

Abstract

This paper introduces Truth as a Trajectory (TaT), a novel interpretability framework that identifies LLM reasoning validity by analyzing layer-wise activation displacements as a dynamic trajectory. Evaluated across Llama-3.1 and Qwen architectures, TaT achieves significant SOTA generalization, outperforming static linear probes and even zero-shot/few-shot base model performance on out-of-distribution reasoning and toxicity benchmarks.

Executive Summary

TL;DR

Existing LLM explainability tools are often "blind" to the process of thinking, looking only at static snapshots of neuron activations. Truth as a Trajectory (TaT) changes the game by treating inference as a continuous path. By analyzing how representations change from layer to layer (displacement) rather than where they are (position), TaT uncovers a universal geometric signature of "truth" that transfers across tasks where traditional methods fail.

Academic Positioning

This work moves beyond the Linear Representation Hypothesis (which assumes concepts are static directions) and situates itself in the emerging field of Transformers as Dynamical Systems. It is a crucial bridge between theoretical "velocity fields" in LLMs and practical, real-world benchmark performance.

Problem: The "Static" Trap in Interpretability

Most mechanistic interpretability researchers hunt for "truth directions" in specific layers. However, this has two major flaws:

Polysemanticity: A single activation vector might represent a "correct answer" AND "the word 'apple'" AND "a formal tone" simultaneously. Linear probes often get confused by these surface-level lexical features.
Task-Specificity: A probe trained to find truth in a math dataset usually fails on a legal dataset because the "geometry of truth" shifts.

The authors argue that reasoning isn't a destination (a point in space) but a trajectory (a refinement process).

Methodology: Mapping the Motion of Thought

The Core Insight: Displacement over Activation

Instead of looking at the hidden state $h_{ℓ}$ , TaT looks at the displacement vector $Δ h_{ℓ}$ : $Δ h_{ℓ} = h_{ℓ + 1} - h_{ℓ}$ This mathematical choice is profound. By subtracting the previous layer's state, they effectively remove the "static background" (like token identity) and isolate what the model is actively adding to the reasoning chain at that specific depth.

Structure of TaT

TaT unrolls the entire computation—every token across every layer—into a single long sequence.

Model Architecture Figure 1: While static embeddings are MESSY, displacements follow smooth, predictable trajectories for correct reasoning (green) and sharp, erratic deviations for incorrect ones (red).

The researchers feed this sequence into a Long Short-Term Memory (LSTM) network. Why an LSTM? Because it is designed to capture sequential dependencies. It "watches" how the model refines its thought process over time.

Experiments: Proving Generalization

The most impressive feat of TaT is its Out-Of-Distribution (OOD) performance.

1. Cross-Task Reasoning

A TaT classifier trained ONLY on the ARC-Challenge dataset was tested on completely different tasks like BoolQ or Hellaswag.

Linear Probes: Accuracy tanked when moving to new tasks (failed to generalize).
TaT: Maintained high accuracy, often beating the base model's own few-shot performance.

2. The Geometry of Toxicity

Toxicity is hard because models often mistake "quoting a bad word" (benign) for "being toxic" (malicious).

Experimental Results Table: Comparison of TaT vs. Probes. Note how TaT (Disp.) consistently yields higher scores on challenging OOD benchmarks like ToxiGen.

TaT successfully distinguished toxic intent from toxic vocabulary. By ignoring the "what" (the toxic token) and focusing on the "how" (the displacement trajectory), it recognized that the model's internal "effort" looks different when it's reasoning through a quote versus generating an insult.

Deep Insight: Why Does This Work?

The success of TaT supports the Privileged Basis Hypothesis. Raw activations are cluttered with information the model is just "carrying along." Layer-wise displacements act as a high-pass filter, leaving only the updates. The authors found that correct reasoning has a "smoother" geometric flow, whereas errors or "spurious correlations" cause the trajectory to jerk or deviate sharply in the high-dimensional manifold.

Limitations & Future Work

Computational Cost: You have to save the residual stream for every layer, which increases memory overhead.
Black-Box LSTM: While the LSTM detects truth, we still don't fully know which specific neurons in the trajectory represent the "aha!" moment of reasoning. The next step is "Trajectory Patching" to find these exact circuits.

Conclusion

Truth as a Trajectory teaches us that if we want to know if an AI is lying or hallucinating, we shouldn't just look at what it says—we should watch how it gets there. The "thought process" is written in the geometry of the displacement, providing a robust, transferable signal for the next generation of safe and interpretable AI.

Find Similar Papers