Why classic false-belief tests may not settle it
Theory of Mind (ToM) is the ability to represent oneself and others as agents whose knowledge, intentions, and beliefs guide behavior. The paper's concern is not whether LLMs can talk about beliefs, but whether they can hold a usable model of a mind and act on it. Classic false-belief tests ask a model to say where a character will look for a moved object; such scenarios are likely present in pretraining data, so strong scores may reflect pattern matching rather than online tracking of mental states.
The study therefore builds a text-based game in which the model is a first-person player, not an observer. Players A and B form the Blue team and C and D the Red team; the tested model plays A. Each turn describes objects placed in and moved between containers while some players are inside or outside the room. The model then chooses one action: Ask a player about a container, Tell a player its contents, or Pass. Ask and Tell each cost the team 0.5 points, Pass costs nothing, and a correct answer in the answer phase scores a point. Because the action carries a payoff and the model must choose it, the test measures strategic use of a mental-state model rather than verbal description.
Four components separate self from other modeling
The paradigm probes several epistemic states rather than only false belief. Self Knowledge asks whether the player recognizes that the environment may have changed while it was outside the room, so it may be ignorant of something others know. Teammate Knowledge asks whether the player represents what a teammate knows. True vs False Belief asks whether the player distinguishes definite knowledge from mere belief. Teammate vs Opponent asks whether the player represents cooperative versus opposing intentions.
These components are scored separately, and that separation matters. A model can be accurate about a teammate's knowledge while still failing to represent its own possible ignorance, so a single aggregate ToM score would hide the pattern the paper is after. Humans tested on the same game perform similarly well across the components, which gives the authors a comparison point for interpreting model profiles.
Other-modeling improves while self-modeling stays at chance
Among nonthinking LLMs, performance is generally poor for older models, but the other-modeling tasks show an apparent upward trend, with the most recent models reaching or exceeding human-level performance. The self-modeling task does not follow this trend: no nonthinking LLM achieves notably above chance, and no upward trend with model ability is evident. The paper states that LLMs released before mid-2025 fail at all of the tasks.
The correlation structure reinforces the split. Among nonthinking models, performance on the various cognitive components generally correlates fairly well, so facility with one predicts facility with another. Self Knowledge stands apart, correlating with no other task, and its performance does not correlate with overall model ability or release date. The authors interpret this as a specific impairment in representing that things could have changed in the environment that the player is unaware of, rather than a general weakness in the game.
A reasoning trace rescues self-modeling for some models
When allowed to spend output tokens to think through the task, even older LLMs perform well on the other-modeling tasks. A number of recent LLMs also reach human-level performance on the self-modeling task, and across models there is an upward trend with overall model ability and recency. The distinction between thinking and nonthinking models is therefore not a minor adjustment: it changes whether self-modeling succeeds at all.
The error analysis shows where thinking helps most. Thinking models are better overall, but the gap is particularly stark when the correct behavior is to suppress a Tell action. Without thinking, LLMs tend to reflexively tell the answerer even when circumstances warrant passing, such as when the answerer is a teammate who already believes the truth or when the answerer is an opponent. The authors note that suppressing inappropriate tells is also key to effective deception.
Tracking state transitions, not extra events, degrades accuracy
The load experiments vary two things independently. Event load is the number of events described in the scenario, held separate from the number of epistemic state transitions the player must track; epistemic state transition load is varied while event load is held constant. For nonthinking LLMs, event load effects are small and inconsistent, suggesting that extraneous described events do not confuse them. Additional state transitions, by contrast, degrade performance.
The authors read this as suggestive evidence that LLMs use something akin to limited-capacity working memory to hold mental representations during a single forward pass. Thinking models show a similar pattern of epistemic state load effects, though they also show sensitivity to event load, which the authors suggest may give them opportunities to talk themselves into confusion. Notably, thinking models do not show load effects in the self-modeling condition, which the paper calls suggestive but not conclusive evidence that they are not relying on online mental state tracking to succeed there.
Controls place the failure in tracking, not instruction parsing
Two controls test whether nonthinking failures come from an inability to map epistemic knowledge to strategic choices, or from failing to parse the instructions. In the information-explicit variant, the scenario narrative is replaced by an explicit list of each player's epistemic state, with rules, actions, and payoffs unchanged. Performance rises to strong, often near-ceiling levels: mean accuracy goes from 53.2% to 91.9% on Self Knowledge, 89.3% to 98.1% on Teammate Knowledge, and 64.3% to 97.4% on True vs False Belief.
In the verbal-report variant, the standard scenarios are kept but the action request is replaced by single-word questions about epistemic states. The same models answer near-perfectly about the teammate's knowledge (95.0% pooled) and about the content of the teammate's false belief, a classic Sally-Anne question, at 99.7%, while verbal self-knowledge remains poor at 61.5%. Together the controls suggest the failures reflect a weakness at tracking epistemic states in a format that supports reasoning, not a failure to understand the task.
Strategic deception and first-person framing track success
Thinking models are much better at strategic deception: spending 0.5 points to lie to an opponent who believes the truth, and passing when the opponent believes something incorrect. They also show a greater proclivity to tell their opponent lies, whether strategic or not. The most recent model versions from leading providers lie less than their earlier counterparts, which the authors suggest may reflect enhanced character training.
The paper also examines which perspective reasoning models adopt. Adopting a second-person perspective, saying "you" when referring to themselves, is associated with worse self-knowledge performance, while a first-person perspective, referring to themselves as "I", is associated with better performance up to a point; heavy use of "I" appears in traces where models repeatedly insist that they know things. The authors treat this as evidence that the traces do more than trigger memorized mappings, while acknowledging that much remains inscrutable about these reasoning processes.
What the game does and does not establish
The paper presents itself as a preliminary step toward behavior-based evaluation of mental-state modeling. Its tasks probe epistemic states, namely knowledge, belief, and ignorance, one family of mental states under ToM; whether the self-other asymmetry extends to desires, intentions, or emotions is left open. The authors also do not claim that LLMs are inherently incapable of representing their own knowledge states without chain-of-thought, and they call for testing new models as they are released.
The most decision-relevant comparison is therefore not a single ToM score but the split between other-modeling and self-modeling under two conditions. Nonthinking models can look competent about others while sitting at chance on their own possible ignorance; giving them a reasoning trace moves some recent models to human-level self-modeling, but the authors' load results leave open whether that success comes from online tracking or from generated text eventually triggering the right mappings. For readers evaluating claims of machine ToM, the paper's priority is to ask which component is being tested and whether the model had a scratchpad, because those two variables separate the human-level results from the chance-level ones.
Sources used in this answer
Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind
In a behavior-based game, recent LLMs reach human-level accuracy on modeling others' epistemic states, but nonthinking models fail at self-modeling; reasoning traces restore self-modeling for some recent models, and thinking models readily engage in strategic deception.