Why can't AI agents just 'figure it out' as they go?
Without a plan, an AI agent has to keep the entire task history in its working memory, which quickly becomes overwhelming. This is called 'entangled context' — the agent must reason over a monolithic history spanning multiple sub-tasks, which increases cognitive load and lets local errors propagate across otherwise independent decisions [1]. One study found that replacing this entangled approach with a task-decoupled plan — where the agent focuses only on the current sub-task — reduced token consumption by up to 82% while improving success rates [1]. That means the agent uses far less computational effort and makes fewer mistakes simply because it isn't trying to juggle everything at once.
Plans also help agents allocate attention to the right things. In a framework called Plan-and-Act, a separate Planner model generates a high-level plan, and an Executor model translates those plans into specific actions. This separation achieved a 57.58% success rate on the WebArena-Lite benchmark and 81.36% on WebVoyager, both state-of-the-art results for text-only agents [3]. The key insight is that generating a plan forces the agent to think ahead about what steps are needed and in what order, rather than reacting step-by-step without a roadmap.
What do checkpoints add that plans alone don't?
Even the best plan can go wrong when the real world doesn't match the agent's expectations. Checkpoints act as safety nets: they let the agent verify that it's on track and, if not, recover locally without restarting the entire task. In a robotic manipulation benchmark, a closed-loop framework that used visual plans as reference inputs and a feedback-driven controller to refine actions and trigger replans as needed improved success rates by 8% over open-loop systems that lacked such checkpoints [5]. The feedback mechanism allowed the robot to detect when it was drifting off course and correct itself in real time.
Checkpoints also make training and evaluation more practical. In the PhysicianBench clinical benchmark, each of the 100 long-horizon tasks was decomposed into structured checkpoints — 670 in total — that captured distinct stages of completion [2]. This allowed researchers to grade partial progress and identify exactly where agents failed, rather than just getting a binary pass/fail. Without checkpoints, a single mistake early in a 27-step task would doom the entire effort, and the developer would have no idea which step went wrong.
How big is the gap between the best agents and reliable performance?
The evidence shows a stark gap: even the best current agents fail more often than they succeed on realistic long tasks. In the PhysicianBench study, which tested 13 proprietary and open-source LLM agents on clinical workflows requiring an average of 27 tool calls, the top model achieved only a 46% success rate, and open-source models maxed out at 19% [2]. That means in more than half of cases, even the most advanced agent couldn't complete a routine clinical task without a mistake. This is not an edge case — it's the norm for long-horizon tasks.
Other studies reinforce this finding from different angles. A framework called Agent S, which uses hierarchical planning and experience retrieval, improved success rates on the OSWorld benchmark by 9.37 percentage points over its baseline — an 83.6% relative improvement — yet still didn't achieve perfect reliability [4]. And in the Spark framework, researchers found that indiscriminately allocating computational resources to all steps wastes budget on trivial steps while failing to guarantee sample quality; their solution was to branch exploration only at critical decision states, which improved success rates with fewer training samples [6]. The consistent message is that plans and checkpoints are not optional extras — they are essential infrastructure for making AI agents work reliably in the real world.
About These Sources
This answer is built on 6 studies (2 peer-reviewed, 4 preprints) — published from 2024 to 2026, 6 from 2024 or later — selected as the most relevant from 10 studies that passed quality screening, drawn from 47 papers retrieved from a database of over 500 million.
Sources used in this answer
Beyond Entangled Planning: Task-Decoupled Planning for Long-Horizon Agents
Proposes Task-Decoupled Planning (TDP), which breaks tasks into a directed acyclic graph of sub-goals, reducing token consumption by up to 82% while outperforming baselines on TravelPlanner, ScienceWorld, and HotpotQA.
PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments
Introduces PhysicianBench, a benchmark of 100 long-horizon clinical tasks (avg. 27 tool calls each) with 670 structured checkpoints; the best proprietary LLM agent achieved only 46% success, and open-source models at most 19%.
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
Presents Plan-and-Act, which separates high-level planning from low-level execution, achieving state-of-the-art 57.58% success on WebArena-Lite and 81.36% on WebVoyager.
Agent S: An Open Agentic Framework that Uses Computers Like a Human
Describes Agent S, an open agentic framework using experience-augmented hierarchical planning, which improved success rate by 9.37 percentage points (83.6% relative improvement) over baseline on OSWorld.
Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation
Proposes CLOVER, a closed-loop visuomotor control framework with feedback-driven replanning, achieving an 8% improvement over open-loop counterparts on the CALVIN robotic manipulation benchmark.
Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning
Presents Spark, which branches exploration only at critical decision states, achieving superior success rates with fewer training samples across diverse long-horizon tasks including embodied planning.