[ICLR 2026] Beyond Scalars: Decoding the "Kinematics" of LLM Reasoning

TL;DR

Evaluating whether an LLM is "thinking" correctly usually involves looking at final probabilities or using an expensive second model as a judge. This paper introduces TRACED, a framework that looks at the physical shape of the reasoning process in latent space. By measuring how much a model "moves" (Displacement) and how much it "turns" (Curvature), the authors can distinguish between a breakthrough and a hallucination with state-of-the-art accuracy.

The Problem: The "Static" Trap

Mainstream evaluation methods treat reasoning as a static output. We either look at the last-token probability (a scalar) or the final text. However, reasoning is a sequence.

• External Assessment: Using a "verifier" model is slow and doesn't scale to real-time inference.
• Internal Assessment: Metrics like Perplexity or Entropy collapse complex thoughts into a single number, ignoring the trajectory of how the model reached the answer.

The authors argue that reasoning is a "Geometric Flow." If a model is hallucinating, its internal states don't just "fail"—they exhibit specific physical patterns, such as circling back on themselves or stalling in one place.

Methodology: Reasoning as a Trajectory

The core of TRACED is treating each step of a Chain-of-Thought (CoT) as a point in a high-dimensional space. The sequence of these points forms a trajectory.

1. Semantic Whitening

Hidden states are often "anisotropic" (bunched up in specific directions). To fix this, the authors project hidden states into the "Vocabulary Space" using the model's own unembedding matrix ($W_U$). This ensures that geometric movement actually corresponds to a change in meaning.

2. Progress vs. Stability

The framework extracts two "Kinematic Signatures":

• Displacement (Progress): How far did the thought travel? High displacement means the model is accumulating certainty and moving toward a conclusion ($M\uparrow$).
• Curvature (Stability): How "jagged" is the path? High curvature indicates "Hesitation Loops"—where the model oscillates between ideas without making progress ($K\uparrow$).

Figure 1: The clear separation between correct (blue) and incorrect (red) reasoning in the Displacement-Curvature plane.

Results: The Laws of "Machine Thought"

The authors discovered a Kinematic Scaling Law. As the number of reasoning tokens ($T$) increases:

• Correct Reasoning scales linearly ($D\propto T$). Each token contributes a constant "velocity" toward the goal.
• Incorrect Reasoning scales sub-linearly ($D\propto \sqrt{T}$). It behaves like a Random Walk, meaning the model is just wandering around the semantic space without a map.

Figure 2: Log-log plot showing the linear vs. square-root growth for correct and incorrect reasoning.

In benchmarks like GPQA (PhD-level science) and MATH, TRACED surpassed traditional "Probing" methods. Because it looks at the whole path rather than just the final token, it is significantly more robust to "long" reasoning chains.

Deep Insight: Mapping Geometry to Cognition

Why does high curvature mean a "bad" answer? The authors mapped these geometric metrics back to cognitive states:

• "Hesitation Loops": High curvature happens when a model's state transitions from Exploration (trying a new path) back to Reflection (doubting the path).
• "Certainty Accumulation": Displacement happens when the model stays in a Certainty state, driving the representation forward in a straight line.

Critical Analysis & Future Work

Strengths:

• Interpretability: Provides a physical, visual explanation for hallucinations.
• Efficiency: No need for labeled data or "judge" models during inference; it's a simple geometric calculation.

Limitations:

• Domain Shifts: Scientific reasoning has "stiffer" (straighter) logic than creative storytelling. While the authors suggest "Centroid Alignment" to fix this, it still requires some reference data for new domains.
• Subspace Sensitivity: The method relies on a logic-heavy subspace ($k=8$). If a model is used for very simple chit-chat, these signatures might be less pronounced.

Conclusion

TRACED proves that geometry is a proxy for truth in LLMs. When a model "knows" where it is going, it moves like a ballistic missile—fast, far, and straight. When it is "lying" or confused, it travels like a drunkard—lots of movement, but very little net progress. This work opens the door for real-time "hallucination sensors" that can stop a model the moment its "curvature" spikes.

[Takeaway]: Valid reasoning follows a "high-progress, low-curvature" trajectory—a fundamental physical signature of machine intelligence.