[CODESCOUT] Reinforcement Learning is All You Need for Code Search Agents
CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents
The paper introduces CODESCOUT, a family of open-source Reinforcement Learning (RL) trained code search agents. Utilizing a simple Unix terminal scaffold and a specialized RL recipe (GSPO), CODESCOUT models achieve SOTA localization performance on SWE-Bench Verified, Pro, and Lite, outperforming much larger models.
TL;DR
Code localization—finding the exact file or function to fix—is the "bottleneck" of autonomous coding agents. While others build complex, language-specific graph tools, CODESCOUT proves that a standard Unix terminal + a smart RL recipe is enough. By training on 39k instances with Group Sequence Policy Optimization (GSPO), their 14B model rivals 32B giants and approaches the performance of frontier closed models like Claude Sonnet.
The "Scaffold" Trap: Why Simple is Better
Most current SOTA code agents use "specialized scouts." They pre-index the repository, build Abstract Syntax Trees (ASTs), or construct complex dependency graphs. While effective, this creates two major problems:
- Language Locking: A Python graph tool won't help you in Rust or Go without massive engineering.
- RL Incompatibility: RL requires thousands of "rollouts" (trial runs); re-indexing a graph for every rollout is computationally ruinous.
CODESCOUT bypasses this by using OpenHands-Bash. The agent has one tool: a bash terminal. It searches using ripgrep, reads with sed, and navigates naturally.
The RL Recipe: Multi-Granularity Rewards
The heart of CODESCOUT is its training methodology. Instead of simple Success/Failure rewards, the authors use a dense multi-granularity F1 score.
1. The Reward Function
The model is rewarded for precision and recall across three levels:
- File Level: Did you find the right file?
- Module Level: Did you find the right class/module?
- Function Level: Did you find the exact function needs changing?
2. Optimization via GSPO
They use Group Sequence Policy Optimization (GSPO), a critic-free algorithm that thrives on long sequences. This allows the model to learn the "intuition" of repository navigation—knowing when to grep for a variable and when to cat a header file.
Better Results with Less Bulk
The experiments on SWE-Bench Verified and Pro reveal a "David vs. Goliath" outcome:
- Efficiency: CODESCOUT-1.7B (a tiny model) beats Qwen-14B base models significantly in F1 scores.
- Precision: Unlike models that return a fixed "Top 5" list, CODESCOUT dynamically decides how many locations are relevant, leading to higher precision which prevents "context rot" in downstream coding tasks.
| Method | File F1 (Verified) | Function F1 (Verified) |
|---|---|---|
| RepoNavigator-32B | 67.75 | 34.09 |
| CODESCOUT-14B | 68.57 | 40.32 |
How Agents Evolve During Training
One of the most fascinating insights is the evolution of tool use. Early in training, the agents use a messy mix of grep, ls, and cat. After RL optimization, the agents "specialize," primarily relying on ripgrep (rg) for discovery and sed for surgical reading. This suggests that the model learns an optimal, "minimalist" strategy for code search.
Final Verdict
CODESCOUT shifts the paradigm from Tool Engineering to Behavioral Training. By proving that a terminal-only agent can outperform graph-navigating agents, it opens the door for truly language-agnostic coding assistants that learn to "think" like senior developers navigating a terminal.
Takeaway: If you want a better coding agent, don't just give it better tools—give it a better RL recipe.