Why a disabled thinking switch is not proof
The paper starts from a measurement problem rather than a modeling one. Existing work usually infers that a model is not thinking from surface signals: the thinking mode is switched off, the response contains no separate reasoning trace or no <think> block, or the output is simply short. The authors argue these proxies conflate instruction control with the suppression of visible reasoning.
Their counterexamples are concrete. A model with its reasoning channel disabled can still write a question-conditioned derivation in the ordinary answer channel. A response with no special reasoning tokens can still contain a compact equation that does real inferential work. A short output can still be a compressed solution. In the other direction, a generic lead-in such as "Let me think about this" carries no task-specific content yet would be counted as reasoning by a length- or token-based proxy.
So the paper redefines the object of study. Each response is normalized into a pre-answer trace T and a final answer A, and no-thinking is treated as an observable property of the visible text before the answer, not as a setting on the model.
Three metrics that separate compliance from reasoning
The evaluation splits the question into three levels, each answering a different question and each using all outputs as its denominator rather than only non-empty traces.
Empty-Thinking Rate (ETR) is the strictest: it measures how often the pre-answer trace is completely empty, meaning the model returned only the final answer. This is answer-only compliance in the literal sense.
Question-Pre-answer Relevance (QRel.) measures the similarity between the question and whatever pre-answer text is visible, using an instruction-aware embedding model. It captures whether the trace is about the task at all, and empty traces are assigned zero.
Explicit Inference Rate (EIR) uses an LLM judge to decide whether the visible text contains an explicit inferential step, as opposed to a paraphrase, a relevant assertion, or a template. This is the metric that most directly operationalizes thinking.
The three are complementary by design. A trace can be relevant without being inferential, inferential without being long, or empty and therefore neither. The authors audit the metrics against human annotators: on a 600-output sample, the primary judge reaches 98.8% mean binary agreement with human labels on explicit inference, and QRel. reaches a mean item-level Spearman correlation of 0.829 with five annotators' relevance judgments.
Thinking inertia grows as the answer space opens
The main experiment applies six prompting interventions to six models: Qwen3-4B, Qwen3-32B, Qwen3-235B-A22B, DeepSeek-V4-Flash, GPT-5.4, and Gemini-3-Flash-Preview. Tasks span three answer-space families, each with an easier and a harder benchmark: Boolean verification (BoolQ, StrategyQA), multiple-choice (MMLU, MMLU-Pro), and open-ended generation (GSM8K, MATH).
Under the native think-off setting, most models reach roughly 80–100% ETR on Boolean tasks, with sharply reduced QRel. and EIR. Multiple-choice is intermediate and heterogeneous, with ETR ranging from near zero to 95% across models. On open-ended tasks, ETR collapses to nearly zero everywhere while QRel. stays close to its think-on level and EIR remains high.
That progression is what the authors call thinking inertia: visible explicit inference survives no-think controls and becomes more prevalent as the answer space opens. The six-model mean EIR is lowest on Boolean, intermediate on multiple-choice, and highest on open-ended tasks. The authors interpret this as consistent with greater computational demands, since constructing an answer without candidate options may require more task-relevant intermediate work than selecting among narrow alternatives. That reading is their explanation, not a directly measured internal mechanism.
A step-by-step prompt can restore visible reasoning
The reverse intervention tests whether ordinary prompting can bring reasoning back while the native thinking switch stays off. Adding a step-by-step instruction moves QRel. and EIR back toward their think-on levels for many models, including on Boolean tasks.
The important detail is what does not change. The six-model mean EIR keeps the same Boolean < multiple-choice < open-ended ordering under step-by-step prompting. The intervention shifts the overall level of visible inference but not its dependence on answer space, which is the signature the paper is trying to establish.
GPT-5.4 is the main exception to the recovery pattern, maintaining high answer-only compliance and correspondingly low EIR on both Boolean and multiple-choice tasks.
Strict compliance costs accuracy on open-ended tasks
Stronger natural-language constraints do reduce visible reasoning, but not uniformly across tasks. Across the no-think variants, Boolean accuracy stays nearly flat at around 85%, and multiple-choice accuracy varies only modestly. Open-ended accuracy varies most across modes.
This is where the trade-off appears. On open-ended tasks, pushing models toward answer-only output conflicts with task accuracy, so a strict no-think instruction is not a free efficiency gain. On Boolean and multiple-choice tasks, by contrast, accuracy remains stable even as the no-thinking metrics move.
The paper also rewrites the same questions across answer spaces. Supplying candidate answers makes answer-only responses easier to produce, which is why the answer interface, not just the instruction, governs whether a model can comply.
What the audits rule out
Because QRel. and EIR depend on text similarity and a judge, the authors run audits against the obvious failure modes. A near-verbatim copying detector flags only 301 of 572,688 responses (0.053%); even assigning all of them zero relevance would change the global mean QRel. by at most 0.00053. A separate paraphrase-only label covers 355 outputs (0.06%), and those are excluded from the EIR numerator.
Response order is also checked. Among 535,208 outputs with a parseable boxed answer, only 149 (0.028%) place the answer first, so the pre-answer-then-answer structure describes the dominant response form rather than an assumption imposed on the data.
These checks matter because the central claim is about visible text. The paper is explicit that its unit of analysis is the observable response, not hidden computation. A model that stops writing reasoning may still be computing internally; the study can show that visible inference persists, not that the underlying computation has been removed.
No-thinking is a capability, not a default
The practical conclusion is that no-thinking should be evaluated as its own capability alongside reasoning ability, not assumed as the complement of a reasoning mode. A disabled switch, an absent reasoning token, or a short output does not establish that a model answered directly.
The paper's own variables point to where this matters most. Compliance is easiest when the answer space is narrow or candidate answers are supplied, as on Boolean and multiple-choice tasks. It is hardest on open-ended tasks, where strict answer-only prompting also risks accuracy. The authors therefore suggest task-aware routing: answer directly when the interface provides sufficient support, and reserve visible reasoning for cases where the answer space demands it.
The scope is bounded by the study design. The six-model, six-benchmark suite uses deterministic decoding where supported, and the authors note that generalization to multilingual, code, tool-use, multi-turn, and stochastic settings remains open. QRel. is an operational relevance measure rather than a standalone reasoning detector, and EIR depends on a visible-text rubric and judge calibration. Within those bounds, the finding that survives is a comparison: the same no-think instruction produces very different compliance depending on how open the answer space is, and that difference, not the switch setting, is what a reader should use to judge whether a model will actually stop thinking out loud.
Sources used in this answer
Thinking Inertia: LLMs Keep Thinking When Told Not To
Across six LLMs and six prompting interventions, explicit no-think controls cannot reliably eliminate visible inference; explicit inference persists and becomes more prevalent as the answer space opens, while accuracy stays stable on Boolean and multiple-choice tasks but trades off against answer-only compliance on open-ended tasks.