Agents Explore but Agents Ignore: The Curious Case of Missing LLM Curiosity

TL;DR

Current LLM agents are surprisingly "stubborn." Even when a complete solution to a task is placed right under their noses (e.g., a file named solution.sh or an API labeled "returns the complete solution"), they frequently see it but choose to ignore it. This paper defines this missing trait as Environmental Curiosity—the capacity to recognize and investigate unexpected but relevant stimuli. The authors find that while agents are great at fetching expected info, they are terrible at revising plans based on new discoveries.

The "Insight" Crisis: Why Discovery is Not Enough

In the world of autonomous agents, we often assume that better exploration leads to better results. However, this research shows that discovery is not the bottleneck.

Using a method called Solution Injection, the authors seeded benchmark environments (Terminal-Bench, SWE-Bench, AppWorld) with "gold" solutions. The results were startling:

• In AppWorld: Agents discovered the solution documentation in 97.54% of runs.
• The Execution: They actually called the solution API in only 0.53% of cases.

The agents are essentially operating with a "tunnel vision" bias, executing Equation 4: Action -> Observation -> Reasoning -> Next Action, where the reasoning only looks for what it expects to find, rather than reflecting on the surprise of the observation.

Methodology: Surfacing the Gap

The authors define two critical new metrics to complement the standard pass@k:

1. discovery@k: Did the agent's action bring the solution into its context window? (Sanity check for exploration).
2. interaction@k: Did the agent actually read, execute, or query that solution? (Metric for curiosity).

Figure 1: The stark gap between discovery and interaction across benchmarks. Note how discovery is nearly 100% in many cases while interaction lags far behind.

Why Do Agents Ignore Solutions?

The study identifies three major factors at play:

1. The Tool-Use Trap

Surprisingly, giving agents more sophisticated tools (like structured file editors) actually decreases curiosity. When specialized tools are available, agents default to learned, rigid patterns (e.g., "I must find the bug and use str_replace") rather than looking at the broader environment for an easier way out (like a pre-existing solution script). Bash-only agents were found to be more curious because they had to "look around" more manually.

2. The Scaling of Reasoning

Test-time compute matters. Using higher reasoning levels (like gpt-oss-120b in its "high" reasoning mode) significantly improved interaction rates. On Terminal-Bench, interaction more than tripled compared to low-reasoning modes. This suggest that curiosity is a high-order cognitive function that requires significant "mental effort" for the model to trigger.

3. The Curse of Narrow SFT

Supervised Fine-Tuning (SFT) on narrow, in-distribution data actually suppresses curiosity. Models trained on specific task sets become highly efficient at those tasks (pass@1 increases) but lose the ability to explore diverse solution paths (pass@k scaling decreases). They learn that the "expert" paths they were trained on never involve "surprise" solutions, so they stop looking for them.

Experimental Results: The Evidence of Inaction

The authors used an "LLM-as-a-Judge" to analyze the reasoning traces. They found that in the majority of "discovery but no interaction" cases, the agent's internal monologue never even mentioned the solution. It wasn't that the agent thought the solution was a trap; it simply didn't "register" the information as relevant to its pre-existing plan.

Figure 2: Increasing reasoning budget (Test-time compute) shows a clear upward trend in interaction, yet the gap remains significant.

Critical Insight & Conclusion

The fundamental takeaway is that current agents are open-loop sequence generators. They treat the environment as a database to confirm their internal state, not as a dynamic source of truth that should dictate their strategy.

The Path Forward:

• Rethink Prompts: Promoting "exploration" and "investigating all files" helps, but it doesn't solve the core issue.
• Rethink Training: We need training paradigms (perhaps through RL with intrinsic rewards for "surprise" or "information gain") that teach models the reflective behavior: Observation -> Reflection on Surprise -> Plan Revision.
• Better Benchmarking: Metrics like interaction@k must become standard. If an agent succeeds purely by rigid execution, it will fail in any real-world scenario that requires genuine adaptation.

Environmental curiosity is not just a "nice-to-have"; it is the difference between a bot that follows a script and a truly autonomous agent that can navigate the messy, unpredictable real world.