What are the options for improving LLM agents?
When you want an LLM agent to get better at a task, you have four main levers: fine-tuning (updating the model's weights on task-specific data), retrieval (giving the model access to external knowledge at inference time), human review (having people check or correct outputs), and adaptive environments (letting the agent learn by interacting with a simulated or real environment that gives feedback). Each has different trade-offs in cost, data needs, and generalization.
Fine-tuning is the classic approach: you take a pre-trained model and train it further on labeled examples. But it requires high-quality data, which is often scarce, and it can overfit—one study found that supervised fine-tuning on synthetic data led to performance collapse on out-of-distribution tasks [2]. Retrieval, on the other hand, doesn't change the model; it pulls relevant information from a database at runtime. A 2024 study comparing the two on knowledge-intensive tasks found that retrieval consistently beat fine-tuning, both for knowledge the model already had and for entirely new facts [3]. The catch is that retrieval only helps if the knowledge is in the database—it doesn't teach the model new skills.
Human review is the most reliable but the most expensive. It's often the bottleneck in real-world deployments. Adaptive environments aim to reduce that cost by having the agent learn from automated feedback, like whether it completed a task or got a reward signal. For example, a system for content compliance review cut manual review costs by 15.3% in a real-world A/B test [1]. So the choice isn't either/or—it's about what you're trying to achieve: inject knowledge (retrieval), teach a skill (fine-tuning or adaptive environments), or ensure quality (human review).
Why adaptive environments beat fine-tuning for generalization and data efficiency
The biggest advantage of adaptive environments is that they let agents learn from interaction, not just static examples. This means they can generalize better to new situations and need far less data. In a 2025 study, environment tuning—where the agent learns by exploring a dynamic environment with corrective feedback—used only 400 problem instances and achieved competitive in-distribution performance while outperforming supervised fine-tuning on out-of-distribution tasks [2]. Fine-tuning, in contrast, collapsed on new tasks because it overfit to the training data.
Another approach, memory-based online reinforcement learning, eliminates fine-tuning entirely. The Memento system stores past experiences in a memory and retrieves them to guide actions, updating the memory based on environmental feedback. It achieved 87.88% on the GAIA benchmark (a suite of general AI assistant tasks) without any gradient updates, and it added 4.7–9.6 percentage points on out-of-distribution tasks compared to training-based methods [5]. This shows that adaptive environments can be both more data-efficient and more robust to distribution shift.
Why does this work? Fine-tuning changes the model's weights to fit a specific data distribution, which can hurt performance on other distributions. Adaptive environments, by contrast, train the agent to respond to feedback in a dynamic context, which encourages more flexible strategies. The 2023 thesis on interactive language-based agents also found that agents trained via reinforcement learning in text-based games could generalize across game families with new objects and instructions [4]. So the evidence consistently points to adaptive environments being superior for generalization, though they require designing a good environment and reward signal, which is non-trivial.
When retrieval and human review are still the right choice
Retrieval is the clear winner when your goal is to inject new factual knowledge without retraining. The 2024 study found that retrieval-augmented generation (RAG) consistently outperformed fine-tuning on knowledge-intensive tasks, and that fine-tuning struggled to learn new facts—even with multiple variations of the same fact [3]. So if you need to update an agent with the latest information, retrieval is cheaper and more effective.
Human review remains essential for high-stakes decisions where errors are costly, but adaptive environments can reduce the burden. The MoE-CL system, which uses continual instruction tuning with a mixture-of-experts architecture, reduced manual review costs by 15.3% in a real-world deployment on Tencent Video [1]. That's a meaningful saving, but it didn't eliminate human review—it just made it more efficient. In domains like medical diagnosis or legal advice, you'd still want a human in the loop.
There's also a hybrid approach: using synthetic data from adaptive environments to improve retrieval or fine-tuning. The Learn-by-interact framework synthesizes agent-environment interaction trajectories and uses them for both in-context learning and training. It improved baseline results by up to 12.2% for in-context learning with Claude-3.5 and 19.5% for training with Codestral-22B across coding, web, and desktop tasks [6]. This suggests that adaptive environments can feed other methods, not just replace them.
About These Sources
This answer is built on 6 studies (2 peer-reviewed, 4 preprints) — published from 2023 to 2026, 5 from 2024 or later, collectively cited 75 times — selected as the most relevant from 7 studies that passed quality screening, drawn from 41 papers retrieved from a database of over 500 million.
Sources used in this answer
Self-Evolving LLMs via Continual Instruction Tuning
MoE-CL, a continual instruction tuning framework with a dual-expert architecture, reduced manual review costs by 15.3% in a real-world A/B test on Tencent Video, demonstrating industrial applicability for self-evolving continual learning.
Don't Just Fine-tune the Agent, Tune the Environment
Environment Tuning, a training paradigm that learns from dynamic environments with corrective feedback, used only 400 problem instances and outperformed supervised fine-tuning on out-of-distribution generalization, avoiding the performance collapse typical of SFT.
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
In a comparison on knowledge-intensive tasks, retrieval-augmented generation (RAG) consistently outperformed unsupervised fine-tuning for both existing and new knowledge, and fine-tuning struggled to learn new facts unless exposed to many variations.
Interactive Language-Based Agents
Interactive language-based agents trained via reinforcement learning in text-based games generalized across game families with new objects and instructions, and a query reformulation agent improved retrieval by interacting with the IR system.
Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
Memento, a memory-based online reinforcement learning agent, achieved 87.88% on GAIA validation and 79.40% on the test set without fine-tuning the LLM, and added 4.7–9.6 percentage points on out-of-distribution tasks via case-based memory.
Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments
Learn-by-interact synthesizes agent-environment interaction trajectories and improves baseline results by up to 12.2% for in-context learning with Claude-3.5 and 19.5% for training with Codestral-22B across coding, web, and desktop tasks, with backward construction providing up to 14.0% improvement.