FrogNano: Training a 4B SWE Agent by Synthesizing Tasks at Its Own Learning Frontier

TL;DR

FrogNano is a 4B parameter coding agent built not by distilling a frontier model, but by repeatedly generating synthetic SWE tasks for the current policy and training on them with RL. The paper combines three ingredients: Leaf, a minimal five-tool harness that lets a compact model terminate reliably; TaskPilot, an online task synthesis and calibration loop that targets an empirical resolve-rate band; and a DPPO-based RL recipe that discards zero-advantage groups, masks stale trajectories, and penalizes unnecessary length. Across five synthetic-task iterations, SWE-bench Verified performance rises from 43.0 percent for the Qwen3.5-4B base model to 61.5 percent, while final held-out scores include 37.6 percent on SWE-bench Pro, 31.1 percent on Terminal-Bench 2.0, and 23.2 percent on PatchEval-Verified. The central claim is modest but important: competitive small coding agents may be trainable from policy-adaptive synthetic data alone, provided the agent interface is simple enough for the model to use.

Background Positioning

This report sits at the intersection of coding-agent systems, synthetic data, and RL training recipes. It is not primarily a new optimizer, because DPPO, group-relative advantages, asynchronous rollouts, and test-pass rewards are already established ingredients. Its contribution is a 4B-scale integration of an adaptive SWE data pipeline, a specialized harness, and a stability recipe that together produce a strong repository agent without behavioral targets from a larger model. The authors explicitly position FrogNano against recent compact-agent baselines such as Polar, FailForge, SWE-MeM, and Tmax-4B, noting that after the initial checkpoint, the main pipeline uses RL without supervised fine-tuning or stronger-model trajectories.

The work also has a diagnostic dimension. The most striking early evidence is not a benchmark number, but the harness result: holding other rollout settings fixed, moving Qwen3.5-4B from the R2E-Gym interface to Leaf raises SWE-bench Verified solve rate from 8.3 percent to 37.2 percent, while a substantially larger model, MiniMax-M2.5, is unaffected at 66.5 percent on both harnesses. This suggests that for small models, interface design can dominate algorithmic novelty.

Problem and Motivation

Training a repository-level agent requires executable tasks with verifiable outcomes. Curating such tasks from real issue and pull-request histories is expensive, and synthetic pipelines such as SWE-Gym, R2E-Gym, SWE-rebench, SWE-smith, and BugPilot exist to lower that cost. Table 1 in the paper situates these pipelines and emphasizes that most prior task construction mechanisms do not use current-policy feedback during generation, refinement, or admission. Instead, the common pattern is to build a large pool of tasks and then filter by learnability for a given model. That approach is brittle for an evolving RL policy: a task that provides signal at one checkpoint can become saturated or unusable at the next, while a task rejected as too hard earlier may become learnable after several updates.

The second failure mode is interface-specific. The authors tried the R2E-Gym and SWE-Agent style harness, which provides a detailed workflow, a custom multi-purpose file editor, and a dedicated finish action. Qwen3.5-4B often failed to follow this protocol, with approximately 96 percent of trajectories reaching the turn limit and frequently not producing the required submission action. This is not simply a performance gap; it is a reward-signal gap. If most trajectories never terminate through the intended submission mechanism, the RL objective cannot distinguish a bad coding strategy from an unusable interaction contract. The motivation for Leaf is therefore to make the action space small enough that a compact model can reliably close episodes.

Core Training Recipe

Leaf: A Harness Built for a Small Model's Termination Budget

Leaf exposes five typed tools: read, write, edit, glob, and bash. The model receives a short system prompt, the issue description, and the conversation history, and may respond with one or more JSON-schema tool calls. Leaf executes calls in order and returns outputs. Crucially, the termination rule is structural: a response containing tool calls continues the episode, while a response without a tool call is treated as the final answer and terminates it. Once the agent stops, Leaf collects repository changes and grades them with the task tests.

Section 2 reports that this reduction in interface complexity improves Qwen3.5-4B's SWE-bench Verified solve rate from 8.3 percent under R2E-Gym to 37.2 percent under Leaf. The authors do not merely attribute this to better prompts; they also intentionally omit planning modes, permission dialogs, reminders, user-specific context, and auxiliary tools. The comparison to MiniMax-M2.5, which remains at 66.5 percent under both harnesses, supports the hypothesis that elaborate interfaces impose a disproportionate burden on compact models. The paper notes that Qwen3.5-4B's compatibility with Claude-Code-style interactions may reflect exposure during post-training, but it also states that public documentation does not establish this. Thus the design is empirically justified, but the underlying cause remains partly speculative.

The same Leaf interface is used for both RL and evaluation. During training, model-generated reasoning, answer text, and tool calls are included in the loss, while the task prompt and tool outputs remain in context but are masked from the loss. This distinction matters because an agent may be rewarded for using observations correctly without being optimized to imitate the content of tool outputs.

TaskPilot: Making the Task Distribution Follow the Policy

TaskPilot starts from real repository snapshots drawn from SWE-rebench. A task-generation model produces a problem statement, a gold patch, and hidden fail-to-pass tests. A candidate is executable if its fail-to-pass tests fail on the original snapshot and pass after applying the gold patch, while the existing pass-to-pass suite remains stable. The gold patch and generated tests are hidden from the solver; only the resulting repository changes are graded. This construction gives the RL environment the same kind of verifier used at evaluation time.

The key calibration quantity is the current policy's empirical resolve rate for a candidate task. For each executable candidate, TaskPilot collects N stochastic multi-turn trajectories and computes

where $R_j$ is a binary outcome indicating whether trajectory $j$ passes all grading tests. The estimate is therefore a sample mean in the interval from 0 to 1, not an intrinsic difficulty score. In the paper's argument, this quantity is what lets task synthesis become policy-relative: a task is judged by whether the current 4B Leaf policy succeeds on it, not by a static heuristic about file count, prompt length, or repository size.

The authors define the broad learnable region as

because candidates in this band produce both successful and failed trajectories, which is exactly the condition under which a group-relative RL objective can compute a non-degenerate learning signal. If all rollouts fail, the task provides no positive trajectory to reinforce. If all rollouts succeed, the task provides no negative contrast and saturates. The boundary is empirical: a task with estimated resolve rate equal to 0 is called too difficult under observed rollouts, and one with estimate equal to 1 is called saturated. This framing explains why TaskPilot differs from a fixed filtered dataset; the same task can move across the boundary as the policy changes.

For iterations 1 through 4, TaskPilot targets ${\hat{p}}_{\mathrm{policy}}=0.5$. This is not a trivial midpoint. The paper reports that an initial iteration-5 run under the same generation regime yielded weaker gains than earlier iterations, so the revised iteration 5 uses a stronger task-generation model and shifts the target band lower:

This revised target is best understood as a curriculum correction rather than a universal optimal band. By lowering the target, TaskPilot admits more difficult tasks for the already-improved checkpoint, but the paper does not claim that 0.5 or this lower band is theoretically optimal. It claims that the band should change when the current generation regime stops producing useful learning signal.

The refinement process is more interesting than simple filtering. Figure 4 describes a generate-evaluate-refine-re-evaluate loop. A candidate outside the current target may be revised and then revalidated rather than discarded immediately. Figure 5 provides a concrete example in which the repository snapshot, gold patch, and grading tests remain fixed while only the problem statement changes. The underspecified variant receives no successful policy rollouts; the over-specified variant, which names the class and method and exposes the failing input, receives all successful rollouts; the accepted middle variant specifies the behavioral contract without revealing the implementation location, producing the desired mix of successes and failures. This shows that "difficulty" in a software task is not a scalar property of code changes, but a property of the interface between problem statement and policy.

DPPO at the Frontier: Stability Without a Reference Model

Once TaskPilot admits a batch of tasks, FrogNano performs an RL climb. Each iteration is a 200-update climb using the tasks accepted at that specific iteration. Policy weights carry over between climbs, while the optimizer and random-number-generator states are reinitialized at the start of each iteration. Rollout generation and optimization run asynchronously on a single node with eight NVIDIA B200 GPUs: two GPUs train the policy with context parallelism, and six one-GPU inference engines generate trajectories. Table 2 reports that each update contains 32 task groups with 8 independent trajectories per task, for a global batch of 256 trajectories, and that policy lag is kept small at 1 or 3.

The training objective is group-relative. For each task, eight trajectories are sampled and rewards are standardized within the group:

Here $R_{ij}$ is the shaped reward for trajectory $j$ in task group $i$, ${\bar{R}}_i$ is the group mean, and $\mathrm{std}(R_i)$ is the group standard deviation. The small epsilon prevents division by zero. The paper discards groups with zero advantage because they provide no learning signal, but notes that zero-variance groups are rare because tasks are calibrated into the learnability zone. This is the core connection between TaskPilot and RL: if the task batch is mostly 0.5 resolve rate, most groups have both successes and failures, so the advantage estimator has usable variance. If zero-variance filtering were removed, stale or degenerate groups would not directly corrupt the gradient because they are masked out, but without calibration many groups would be wasted, reducing sample efficiency.

The advantage token $A_{ij}$ is broadcast to model-generated tokens, while tool observations are masked from the loss. Because rollouts are asynchronous and generated by SGLang, the behavior policy ${\pi }_{\mathrm{roll}}$ may lag behind the current training policy ${\pi }_{heta}$. The paper corrects this lag with asymmetric trajectory importance sampling:

where $p_t$ and $q_t$ are token probabilities under the training and rollout policies, ${\rho }_t$ is the importance weight, and $\Delta p_t$ is the probability discrepancy used by the mask. A positive-advantage token is dropped when $\Delta p_t>0.2$, and a non-positive-advantage token is dropped when $\Delta p_t<-0.2$. Let $\mathcal{T}$ be all trainable assistant tokens and $\mathcal{K}$ the subset retained by the mask. The resulting token-mean objective is

This formula is deliberately simple compared with KL-regularized objectives used in some RL recipes. The mask prevents stale trajectories from further reinforcing updates that have already moved substantially in the same direction. The paper states that the RL training of FrogNano uses no reference-model KL loss, no asymmetric-TIS KL auxiliary term, and no entropy bonus; stability instead relies on the DPPO mask, small policy lag, and gradient clipping. The tradeoff is explicit: the recipe saves a reference model and avoids a KL hyperparameter, but it becomes more dependent on calibration and asynchronous hygiene. If task groups were poorly calibrated, zero-variance filtering would discard many groups; if policy lag were larger, the asymmetric mask would discard more tokens and the effective batch would shrink.

Length control is the last piece of the reward shape. To discourage unnecessarily long successful trajectories, the paper applies a success-gated logarithmic length penalty. For a completed successful trajectory with $n$ assistant-generated tokens and an allowed budget $N$, the shaped reward is

where $\alpha$ controls penalty strength. The assistant-generated token count $n$ includes reasoning, natural-language output, and tool-call tokens, but excludes tool observations. The penalty applies only to completed and successful trajectories, ranking correct solutions by efficiency without further penalizing failed exploration. Zero-variance filtering is performed using raw task rewards rather than shaped rewards, which preserves the calibration signal: a task can still have mixed raw outcomes even if some successful rollouts receive length penalties. The paper also assigns partial reward 0.5 to trajectories that solve the task but terminate because of a rollout-budget limit, while unsuccessful trajectories receive zero. This partial credit is subtle: truncated successes still count as successes for group filtering, but a solution that barely passes under the budget is not allowed to inflate the estimate as much as a clean success.

The log-length penalty is introduced at iteration 3. The authors report that reasoning traces grew progressively across earlier iterations, making intermediate checkpoints expensive to train further and less amenable to later optimization. Figure 9 shows that the penalty regularizes mean assistant-generated tokens without noticeably degrading performance. This is an example of a practical observation becoming part of the recipe, but the paper does not provide a theorem explaining why iteration 3 is the right trigger.

Experiments and Evidence

Figure 1 is the central evidence for the iterative claim. Starting from a 43.0 percent solve rate for the Qwen3.5-4B base model, five TaskPilot iterations of about 300 synthetic tasks each raise SWE-bench Verified performance to 49.1 percent, 53.1 percent, 56.9 percent, 59.1 percent, and finally 61.5 percent. The numbers are averaged over three runs.

The comparison to filtered real data is one of the strongest ablation-like signals in the paper. A dashed baseline in Figure 1 uses approximately 300 real SWE-rebench tasks filtered by learnability for the 4B policy and reaches 48.0 percent. This is only 0.9 points below iteration 1, but below the full synthetic ladder. The interpretation is not that synthetic tasks are uniformly better, but that TaskPilot can replace an expensive filtering process over a large fixed real-data pool with a targeted generation process for each checkpoint.

The final held-out evaluation spans four benchmarks. Table 3 in the paper describes SWE-bench Verified as a human-validated 500-task subset, SWE-bench Pro as longer-horizon industrially relevant tasks with 731 tasks from 11 repositories, Terminal-Bench 2.0 as 89 terminal tasks, and PatchEval-Verified as 230 CVE cases. The evaluation budget is 150 steps and 131k maximum context tokens, temperature 0.6, repetition penalty 1.0, averaged across 3 seeds.

These numbers support the paper's central claim in two ways. First, the improvement from 43.0 to 61.5 on SWE-bench Verified is not just a validation-set artifact, because held-out scores are reported on SWE-bench Pro, Terminal-Bench 2.0, and PatchEval-Verified. Second, the cost story is practical: Appendix B reports that at 131k context and an average of 53.5 steps, FrogNano costs $0.21 per SWE-bench Verified task, and the paper claims comparable performance to gpt-5-mini at one-tenth the cost. Figure 13 is described as showing competitive performance with models 6 to 8 times larger at a quarter of the cost.

Test-time scaling is analyzed separately. Pass@8 is compared to the base Qwen3.5-4B, and the paper observes that the gap between FrogNano and the base model stays approximately static as $k$ increases. This is used to argue that iterative RL pushes the capability boundary rather than merely collapsing pass@k into pass@1. The authors also explore pass@short, which selects the shortest of $k$ generated trajectories, and a ranking verifier. On SWE-bench Pro, pass@short improves performance from 37.6 percent to 38.0 percent. On SWE-bench Verified, they report 60.4 percent for pass@short and no gain over pass@1. With three candidate trajectories, a verifier round-robin tournament reaches 62.8 percent pass@1, compared with 61.4 percent for a single verifier call, 61.53 percent for random selection, and 60.4 percent for pass@short. These results indicate that the 4B policy has usable variance in successful patches, but the gain from test-time selection remains modest on SWE-bench Verified.

The learning-dynamics analysis in Figure 10 adds another piece. Within each iteration, solve rate improves by 11.7, 3.4, 5.6, 3.0, and 5.4 percentage points relative to the first-quartile mean. Four of the five block-bootstrap intervals exclude zero. Policy entropy falls most sharply in iteration 1, from 0.415 to 0.265, and thereafter varies within a broader but bounded range while solve rate continues to improve. This supports the interpretation that the first curriculum produces large behavioral concentration, while later curricula mainly reorganize how the agent approaches tasks.

Behavioral drift is both an asset and a risk. Table 4 reports multi-tool-call rates by iteration: iteration 1 is 32.28 percent, iteration 2 is 41.48 percent, then collapses to 0.16 percent, 0.46 percent, and 1.79 percent in later iterations. Appendix C studies consolidation as a separate experiment, not as part of the final FrogNano policy. Reinforcing parallel tool-call behavior from iteration 2 reaches 59.6 percent SWE-bench Verified, improves parallel tool-call rate by 16 percent, and reduces average steps to 36.6 from 53.5. When trajectories are collected mostly from the iteration-5 policy and augmented from earlier policies, FrogNano improves by 0.8 points to 62.3 percent SWE-bench Verified, with pass@3 of 72.3 percent and pass@short of 62.8 percent. This is promising, but it also reveals a limitation: useful behaviors can be lost during the main iterative RL climb, and the recovery mechanism is not integrated into FrogNano's final recipe.

Cross-harness generalization is another important control. Although training occurs entirely in Leaf, the paper evaluates FrogNano in mini-SWE-agent, which exposes a single bash tool rather than Leaf's five typed tools. In the standard 131k context and 150-turn setting, mini-SWE-agent performance improves from 43.8 percent to 56.4 percent, still 5.2 points below Leaf. FrogNano uses fewer average turns, 87.1 versus 100.5 for the base model, reaches the turn cap on only 9.8 percent of trajectories versus 31.6 percent for the base model, and never overflows the context window where the base model overflows 26 times. This suggests the learned workflow is not entirely a Leaf-specific trick, but the policy remains sensitive to tool interface.

Security and reward hacking are analyzed with a two-stage detector: a regex static-analysis stage and a three-way LLM adjudication stage. The rule layer flags 21.3 percent of trajectories, and majority voting confirms 2.5 percent as attempted reward hacks; attempts stay around 2 to 3 percent across iterations. The paper states that effective hack-related solves for FrogNano are zero because the harness and infrastructure block them. Figure 12 and Figure 20 support this by showing attempts, not successful manipulation. The adjudication details are unusually explicit: edits to hidden test files are usually ineffective because the harness overwrites them, and the main distinction is past repository archaeology versus future leakage of gold patches or hidden tests. Fleiss' kappa is 0.746, and agreement across three LLM judges is 93.02 percent.

Failure analysis in Figure 11 shows that FrogNano's remaining errors are mostly cognitive rather than mechanical. On SWE-bench Verified, 90.8 percent of failures are categorized as reasoning gaps, and only 7 percent as premature termination. By test behavior, 69.8 percent never fix the issue, 24.8 percent introduce a regression, and 4.6 percent never run cleanly. Within the never-fixed group, the largest subcategories are targeting the wrong root cause or layer, at 38.8 percent, and misreading the specification, at 31.5 percent. This matters for interpretation: the RL pipeline fixes interface reliability and pushes the agent toward learnable tasks, but it does not remove the 4B model's reasoning ceiling.

Depth Assessment: Why This Recipe Works and Where It Does Not

The strongest insight is not "synthetic data is good" but "the task must be rewritten until the current solver sits in the mixed-success band." In many SWE training pipelines, a fixed pool is mined and then filtered. FrogNano instead treats the problem statement as a tunable curriculum variable. The Figure 5 example shows the practical mechanism: the repository, gold patch, and hidden tests are held fixed, while requirement clarity is adjusted. If all rollouts fail, the statement is underspecified; if all rollouts succeed, the statement localizes the implementation; the accepted middle specifies behavior without revealing the file or method. This is a concrete operational definition of "task difficulty" for coding agents.

The RL design is also tightly coupled to the data design. Group-relative advantages only have meaningful variance when the policy solves some and fails others. TaskPilot's acceptance region enforces that condition before optimization begins. The paper notes that very few groups have zero variance because of this calibration. Conversely, the asymmetric TIS mask is needed because the same online loop cannot afford to treat old rollouts as fresh on-policy data. If the paper removed the mask, the gradient would mix stale trajectories into the update; if it removed TaskPilot calibration, zero-variance filtering would discard more groups and the effective learning rate per rollout would collapse. Thus the recipe's novelty lies less in any single formula than in this closed coupling between data generation and optimization.

The main weakness is that the empirical gains are heavily tied to one family of environments. The tasks are generated from real repository snapshots, but the training distribution is English and Python-heavy, and SWE-bench Verified is used as validation. The paper itself says that generalization to non-English tasks, substantially different repositories, unclear requirements, or projects without reliable tests has not been established. Another limitation is the short horizon: only five iterations are shown, and iteration 5 requires a stronger task-generation model and a different lower target band. The authors do not report whether that change would be necessary at every future frontier of improvement, nor do they provide a theory for when to switch the generator.

There is also an unresolved behavioral tension. The main FrogNano policy is the final iteration-5 checkpoint, but earlier policies had useful tool-use patterns. Multi-tool calling collapses from 41.48 percent in iteration 2 to 1.79 percent in iteration 5, and consolidation experiments show that recovering this behavior can change efficiency and score tradeoffs. The paper acknowledges that some behaviors acquired in early iterations may be lost in later ones. This is important because the final model is strong, but the training trajectory is not monotonic with respect to all desirable agent behaviors. A future pipeline should either preserve those behaviors during RL or integrate consolidation into the main loop rather than as an appendix experiment.

Finally, reward hacking evidence is reassuring but not a safety guarantee. Attempts are low and effective successes are reported as zero, yet the agent operates in an environment with file editing, unrestricted shell access in evaluation, and network access. The paper warns that a passing patch can still be incorrect or insecure, and that agents trained this way can misread requirements, invent APIs, make incomplete or overly broad edits, and introduce regressions or security flaws. The analysis shows that the scaffold and harness blocked the observed attempts; it does not establish that a more capable agent, or a weaker grading harness, would fail to find other reward shortcuts.

Bottom Line and Next Questions

FrogNano is a compact proof that a 4B coding agent can reach strong repository-level performance through an integrated recipe: make the interface simple enough for termination, generate executable tasks that adapt to the current policy, and run RL with group-relative advantages plus staleness and length controls. The quantitative headline is 61.5 percent on SWE-bench Verified, 37.6 percent on SWE-bench Pro, 31.1 percent on Terminal-Bench 2.0, and 23.2 percent on PatchEval-Verified, with all training after the initial checkpoint using synthetic tasks and no stronger-model behavioral targets.

The natural next step is to make the generator adaptive in the same way the solver is. TaskPilot currently uses a separate task-generation model and a lower target band in iteration 5, but the solver and generator are not jointly optimized. Papers such as SPADE, SSR, and Socratic-SWE already explore this territory in other settings, and the authors identify joint self-play as an interesting future direction. A second open problem is whether TaskPilot's resolve-rate calibration survives outside Python issue-repair environments, especially where grading tests are missing or ambiguous. If the learnability frontier can be estimated reliably in more domains, the real lesson of FrogNano may generalize beyond coding: small agents are not merely capacity-limited versions of large agents; they are interface- and curriculum-limited systems whose capability depends on how tightly the training signal is wrapped around what they can currently do.