LLMs as Nondeterministic Causal Models: A Simpler Path to Counterfactual Reasoning

TL;DR

Generating "what if" scenarios (counterfactuals) for LLMs usually requires messy hacks into the model's sampling code. This paper argues that if we treat LLMs as nondeterministic causal models rather than deterministic machines, the math simplifies beautifully: the "counterfactual" output is often just a standard output from a new prompt. This allows us to generate counterfactual explanations for even the most restricted black-box models like GPT-4.

The "Literal" Trap: Why Current Methods are Overcomplicated

When we ask an LLM a question and get an answer, we often want to know: What would it have said if I changed one word in my prompt?

Previous researchers (Chatzi et al., Ravfogel et al.) treated LLMs as deterministic systems hidden behind a mask of randomness. They focused on "Literal Interpretation"—modeling the exact Pseudo-Random Number Generator (PRNG) and the specific seed used during inference. This is a technical nightmare. If you don't have the seed, you can't generate the counterfactual. Since commercial LLMs are stateless and black-boxed, these methods are practically useless for the average developer.

The Insight: Embracing Nondeterminism

The author, Sander Beckers, suggests an Idealized Interpretation. Instead of obsessing over the "physics" of the computer's CPU clock or random seeds, we should treat the sampling of a token as a truly stochastic event.

The Formal Collapse of the Causal Hierarchy

In causal inference, there is a famous "Hierarchy" (Pearl's Ladder of Causation):

1. Association (Seeing)
2. Intervention (Doing)
3. Counterfactuals (Imagining)

The paper's breakthrough is a proof showing that for LLMs, the hierarchy collapses. Because LLMs are autoregressive—each token depends on all previous tokens—the "actual" world (the prompt you gave) provides zero information about the "counterfactual" world ( a different prompt).

The result? The counterfactual probability $P^{\ast }(Y^{\ast }|Y,X,X^{\ast })$ is mathematically equivalent to the standard probability $P(Y|X^{\ast })$. This is the Simple Semantics.

Methodology: Selection Functions over Gumbel-Max

Previous methods used the Gumbel-Max trick to ensure "Counterfactual Stability" (CS)—the idea that the counterfactual output should be "close" to the original output. Beckers argues that CS is just one type of bias we might want.

By using Selection Functions, we can pick and choose our biases without needing a deterministic model:

• Unbiased Selection: Just run the model again.
• Stable Selection: Filter results to keep them similar to the original.
• Explanation Selection: Purposely look for results that are different to explain why the original answer occurred.

Table 1: Showing how the Nondeterministic approach avoids the arbitrary biases (Line 6) introduced by the Gumbel-Max method.

Why This Matters for the Future of XAI

The most exciting application is in Counterfactual Explanations. If an LLM denies a loan application, a counterfactual explanation would be: "If your income had been $500 higher, the model would have approved you."

To find these "flipped" outcomes, we actually don't want counterfactual stability; we want to find how the model's behavior changes. This paper provides the theoretical "green light" to use simple sampling to find these explanations across any LLM, regardless of its architecture.

Conclusion

By moving from a literal, implementation-heavy view to an idealized, nondeterministic view, Beckers has democratized counterfactual reasoning for LLMs. We no longer need to peek under the hood of the sampling engine; we just need to understand the causal structure of the conversation itself.

Limitations: While this works perfectly for LLMs due to their specific autoregressive structure, it doesn't immediately solve counterfactuals for Markov Decision Processes (MDPs) where state information is lost. That remains the next frontier.