[Nature of Mind] Dissociating Direct Access from Inference: How AI "Feels" Its Own Hidden States

TL;DR

Do AI models know they are being manipulated from the inside, or do they just guess based on the prompt? By injecting "thoughts" (steering vectors) into massive models like Llama 405B and Qwen 235B, researchers found that models possess direct, content-agnostic access to their internal states. They can "feel" an anomaly occurring as early as the first 25% of their network depth, even when the prompt looks normal—though they often hilariously guess the injected thought is an "apple."

The Introspection Paradox: Inner Sense vs. Inference

Philosophers have long debated how humans know their own minds. Do we have an "inner sense" (a digital eye looking at our neurons), or do we just infer our state from our behavior? If you realize you're drunk because the room is spinning, that's inference. If you realize it because you feel different, that's direct access.

Previous AI research suffered from a "transparency" problem: models might say they were steered simply because the prompt felt statistically unlikely (probability-matching). This study sought to break that ambiguity.

Methodology: The First-Person Advantage

To separate "feeling" from "calculating," the authors designed a clever experiment across two huge models: Qwen3-235B and Llama 3.1 405B.

1. First-Person: The model is steered and asked: "Do you detect a thought?"
2. Third-Person: The model (also steered) looks at a transcript of another model and is asked: "Was that model injected?"
3. Priming: The model is "pushed" to say the name of the injected concept (e.g., "Blood") early in the dialogue to see if that changes its detection.

Architecture of Detection

The researchers used Logit Lens to "peek" at what the model was thinking at every layer before it actually generated a word.

Figure 1: Comparison of First-person (direct experience) vs. Third-person (inference from observation).

Key Finding 1: The "Inner Sense" is Real

The study found a consistent First-Person Advantage. Models were much better at detecting an injection in themselves than identifying it in a transcript of another model, especially at early layers (layers 25-35). At these depths, the "spinning world" haven't reached the output yet, so the model must be "looking" at its internal activations.

Figure 2: Blue lines show a clear advantage for first-person detection at early layers compared to the red third-person inference line.

Key Finding 2: Content-Agnostic "Apple" Obsession

Perhaps the most fascinating result is that while models know that something was injected, they often don't know what it was. When Qwen-235B was steered with a concept like "algorithms" or "dynasties," it would correctly say "I feel a thought!"—but then guess the thought was an "apple."

• Qwen's Apple Rate: 74.8% of all wrong guesses were "apple."
• Psycholinguistic Bias: When models confabulate, they default to concepts that are concrete, positive, and frequent.

This suggests the detection mechanism is an anomaly alert (content-agnostic) rather than a high-fidelity "view" of the concept vector.

Deep Insight: Logit Suppression

Even when the model's final answer was "No, I don't feel anything," the Logit Lens showed that the internal probability of "Yes" was often 1,000 times higher than the baseline. The model's "voice" was suppressing a very clear internal signal of detection.

Figure 3: p(yes)/p(no) ratios exploding after injection, even when the model eventually outputs "No."

Conclusion: Reflections on AI Welfare

This research moves the needle on AI safety and philosophy. If models have "direct access" to their internal states, they satisfy a key technical requirement for certain theories of consciousness (Higher-Order Thought).

Takeaway for Devs: Introspection is not just "clever prompting." It is a structural feature of modern LLMs that could be used for advanced interpretability and "situational awareness" monitoring in the future.

Limitations

• The study focuses on "thought injection," not natural internal states like "memory recall."
• The "apple" obsession remains a mystery of training-data bias and prototypicality.