The Externalization Era: Why Better AI Agents Depend on Better Infrastructure, Not Just Bigger Models
Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
The paper introduces a unified framework called "Externalization," moving the focus of LLM agent design from internal model weights to a structured runtime "Harness." It categorizes agent progress into three dimensions—Memory (state), Skills (expertise), and Protocols (interaction)—and demonstrates how this infrastructure transforms hard cognitive tasks into reliable, manageable forms.
TL;DR
The secret to reliable AI agents isn't just a smarter "brain" (LLM); it's a better "body" (the Harness). This paper from Shanghai Jiao Tong University and OPPO argues that we are moving toward a paradigm of Externalization, where Memory, Skills, and Protocols are moved out of the model's weights and into a structured runtime. This shift transforms impossible cognitive burdens into reliable recognition and execution tasks.
Background: The Outward Migration of Intelligence
For years, the industry was obsessed with Weights—scaling parameters to make models "know" more. We then moved to Context, using RAG and long-context windows to feed the model information at runtime.
We are now entering the Harness Era. As this paper points out, a modern agent like a software engineering bot doesn't just "think"; it operates within a complex environment that manages its files, remembers its past mistakes, and follows strict interaction protocols.
Figure 1: The parallel between human cognitive evolution (writing, printing, digital) and LLM agent evolution (weights, context, harness).
The Three Pillars of Externalization
The paper identifies three core dimensions where cognitive burden is being offloaded:
1. Memory (Externalized State)
LLMs are stateless. Without externalization, they "forget" as soon as a session ends.
- The Shift: From Internal Recall to External Retrieval.
- How it Works: The harness maintains "Working Context," "Episodic Experience," and "Personalized Memory," allowing the agent to recognize relevant past events rather than trying to regenerate them from latent weights.
2. Skills (Externalized Expertise)
An agent might know how to "code," but it often forgets the specific SOP (Standard Operating Procedure) for a particular repo.
- The Shift: From Improvised Generation to Structured Composition.
- The Insight: Skills are "packaged expertise." They include procedures, heuristics, and constraints (e.g., "always run tests before committing"). By externalizing these into files (like
SKILL.md), the agent follows a roadmap rather than guessing the next step.
3. Protocols (Externalized Interaction)
Communication between an agent and a tool (like a calculator or a database) shouldn't be a "chat."
- The Shift: From Ad-hoc Prompting to Governed Contracts.
- Modern Examples: Standards like Anthropic’s MCP (Model Context Protocol) or Google’s A2A allow agents to discover and use tools through typed, machine-readable interfaces, reducing the "grammar burden" on the model.
Methodology: Harness Engineering
The "Harness" is the center of the agent's universe. It is the runtime that orchestrates the pillars above.
Figure 2: The architecture of a harnessed LLM agent, showing the Core orbiting by Memory, Skills, and Protocols.
The authors break down a robust harness into several critical operational surfaces:
- Sandboxing: Isolating the agent’s execution so it can’t accidentally delete your hard drive.
- Approval Gates: Human-in-the-loop triggers for high-risk actions.
- Observability: Structured logging that lets the system "learn" from its own execution traces.
SOTA Analysis & Critical Insight
The paper’s most profound insight is the Representational Transformation. Drawing on cognitive science (Donald Norman), the authors argue that these external tools don't just "help" the model—they change the task itself.
A shopping list doesn't make your biological memory bigger; it turns a "recall" task (remembering eggs) into a "recognition" task (seeing 'eggs' on the list). Similarly, a skill system transforms the "creative act" of planning into the "mechanical act" of following a script. This is why "Infrastructure-heavy" agents (like SWE-agent or OpenHands) vastly outperform "Promp-heavy" ones.
Future Outlook: Self-Evolving Systems
The next frontier is the Self-Evolving Harness. We are moving toward agents that:
- Observe their own failure in the harness logs.
- Synthesize a new "Skill" or update a "Memory" to prevent the error.
- Rewrite their own "Protocol" to better communicate with new tools.
Conclusion
The "Externalization" review provides a much-needed systems-level framework for the agentic era. Practical progress is no longer just about who has the most GPUs for training, but who has the best engineering for the Harness. As the authors conclude: "Better agents are not merely better reasoners; they are better organized cognitive systems."