Why do agents keep repeating mistakes in long tasks?
The old assumption was that a large language model could just keep a running conversation and remember what it did. But in practice, that fails: most frameworks rely on linear prompt concatenation or shallow memory buffers, which leads to brittle performance, frequent hallucinations, and poor long-range coherence [1]. In other words, the model forgets or confuses earlier steps, so it repeats the same error or loses the thread entirely.
The scale of the problem is stark: in a standard benchmark like Towers of Hanoi, even recent models derail after at most a few hundred steps [2]. That means for any task longer than a few hundred operations, you cannot trust the model's internal memory to keep it on track.
What actually works: structured memory and step-by-step checks
The first fix is to give the agent an external, structured memory instead of relying on the model's context window. The Task Memory Engine (TME) uses a hierarchical Task Memory Tree where each node stores the input, output, status, and sub-task relationships of a step [1]. The prompt is then generated dynamically based on the active node path, which improves execution consistency and contextual grounding [1]. In plain terms, the agent always knows exactly where it is in the task and what it has already done, so it doesn't repeat a step or lose track of dependencies.
The second fix is to verify each step, not just the final answer. A 2023 study compared process supervision (checking each intermediate reasoning step) with outcome supervision (checking only the final result) and found that process supervision significantly outperformed outcome supervision on the challenging MATH dataset [3]. The process-supervised model solved 78% of problems, a clear jump in reliability. The takeaway: catching mistakes early prevents them from compounding into repeated failures later.
Can agents scale to million-step tasks without errors?
Yes, but it requires a radical change in architecture. The MAKER system solved a task with over one million LLM steps with zero errors by breaking the task into extremely small subtasks, each handled by a focused microagent [2]. This extreme decomposition makes each step simple enough to verify, and an efficient multi-agent voting scheme corrects errors at every step [2]. The result: error correction becomes cheap and effective, so mistakes don't accumulate.
This approach is fundamentally different from trying to make a single model better at long tasks. Instead of relying on continual improvement of current LLMs, massively decomposed agentic processes (MDAPs) may be the way to scale to organizational and societal-level problems [2]. The trade-off is complexity: you need to orchestrate many microagents and a voting mechanism, which adds overhead but pays off in reliability.
About These Sources
This answer is built on 4 studies (all preprints) — published from 2023 to 2025, 2 from 2024 or later — selected as the most relevant from 4 studies that passed quality screening, drawn from 33 papers retrieved from a database of over 500 million.
Sources used in this answer
Task Memory Engine (TME): Enhancing State Awareness for Multi-Step LLM Agent Tasks
The Task Memory Engine (TME) uses a hierarchical Task Memory Tree to track each step's input, output, status, and sub-task relationships, and dynamically synthesizes prompts from the active node path, improving execution consistency and contextual grounding in multi-step agent tasks.
Solving a Million-Step LLM Task with Zero Errors
MAKER, a system using extreme decomposition into microagents with multi-agent voting for error correction, solved a task with over one million LLM steps with zero errors, demonstrating that massively decomposed agentic processes can scale far beyond current LLM limits.
Let's Verify Step by Step
In a comparison on the MATH dataset, process supervision (feedback per intermediate step) significantly outperformed outcome supervision (feedback on final result), with the process-supervised model solving 78% of problems, and active learning further improved efficacy.
Chaining AI Agents in PaaS Architectures for Multi-Step Workflow Automation
This paper describes architectural designs and orchestration strategies for chaining multiple AI agents in Platform-as-a-Service (PaaS) environments to automate multi-step workflows, with case studies in IT incident response and automated remediation.