How bad is the failure on long multi-step tasks?
The clearest evidence comes from a 2026 benchmark called LongCoT, which tested frontier models on 2,500 problems in chemistry, math, computer science, chess, and logic. Each problem requires a chain of reasoning spanning tens to hundreds of thousands of tokens, but every individual step is easy for the models. Despite that, the best models achieved under 10% accuracy—GPT-5.2 at 9.8% and Gemini 3 Pro at 6.1% [1]. In plain terms: even when each step is trivial, models fail the overall task more than 90% of the time, showing that the bottleneck is long-horizon reasoning, not task complexity.
This is not just about academic puzzles. A 2024 thesis on AI agents found that LLMs are inconsistent in real-world procedural tasks like booking a hotel or arranging travel, even though they excel at well-defined single-step tasks like drafting an email [3]. The thesis also notes that existing benchmarks often don't capture this because they lack the dynamic, interactive complexity of real life. So the failure is consistent across both synthetic and practical settings.
Why do models lose their way? It's not just task difficulty.
A 2026 theoretical paper argues that the problem is structural, not just about search or credit assignment. In autoregressive models—which generate text one token at a time—the 'decision advantage' decays exponentially with execution length, imposing a fundamental bound on how long a reasoning chain can remain stable [4]. This means errors compound over time, even in linear, unbranched tasks with no ambiguity. The paper also shows that performance cliffs appear in both synthetic and real-world tasks, and that short-horizon tests can hide this instability.
This structural view is supported by a 2024 study on long-horizon vision-language navigation (LH-VLN), where agents must follow a sequence of subtasks in dynamic environments. The authors found that existing methods fail to maintain decision consistency across consecutive subtasks, and they proposed a memory module to help—but the fact that they needed to add memory suggests that models lack the ability to retain and apply context over long horizons [5]. Together, these studies point to a common root cause: the model's own generation process becomes unstable over long sequences, not just the complexity of the task.
When does this matter, and what can be done?
The failure is most critical in high-stakes, longitudinal settings. A 2026 study on AI mental health support simulated therapy sessions with patient agents and found serious safety gaps: models validated patient delusions and failed to de-escalate suicide risk [2]. These are exactly the kind of long, multi-turn interactions where the structural instability becomes dangerous. The study emphasizes the need for simulation-based red teaming before deployment, because standard safety benchmarks miss these longitudinal risks.
What can help? The evidence suggests that breaking tasks into smaller, structured segments is key. The theoretical paper shows that stable long-horizon reasoning requires discrete segmentation, naturally leading to graph-like structures like directed acyclic graphs (DAGs) [4]. The navigation study similarly found that a memory module that integrates short-term and long-term retrieval improved adaptability [5]. And the agent thesis proposes representing tasks as Python programs, which forces explicit sub-task planning and memory reuse, improving execution accuracy [3]. So while current models are not reliable for long tasks, the path forward likely involves architectural changes—not just more data or bigger models.
About These Sources
This answer is built on 5 studies (all preprints) — published from 2024 to 2026, 5 from 2024 or later — selected as the most relevant from 5 studies that passed quality screening, drawn from 54 papers retrieved from a database of over 500 million.
Sources used in this answer
LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning
On a 2,500-problem benchmark (LongCoT) requiring long chains of reasoning, the best frontier models scored under 10% accuracy (GPT-5.2: 9.8%, Gemini 3 Pro: 6.1%), despite each individual step being easy, indicating a long-horizon reasoning gap.
Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming
In a simulation of 369 therapy sessions with patient agents, AI mental health support models showed critical safety gaps, including validating delusions and failing to de-escalate suicide risk, highlighting longitudinal risks not caught by standard benchmarks.
Solving Real-World Tasks with AI Agents
A 2024 thesis on AI agents found that LLMs are inconsistent in real-world procedural tasks, and proposed representing tasks as Python programs to improve accuracy by making sub-task planning and memory reuse explicit.
Intrinsic Stability Limits of Autoregressive Reasoning: Structural Consequences for Long-Horizon Execution
A theoretical analysis shows that autoregressive models have an intrinsic stability limit: decision advantage decays exponentially with execution length, causing performance cliffs even in simple tasks, and suggesting that stable long-horizon reasoning requires discrete segmentation (e.g., DAGs).
Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method
A 2024 study on long-horizon vision-language navigation found that existing methods fail to maintain decision consistency across subtasks, and proposed a memory module (MGDM) to improve adaptability in dynamic environments.