WHALE: A Simple Recipe That Trains the Harness and the Model Together

Core Snapshot

WHALE, or Weight-Harness Alternating Learning, treats an agentic language-model system as a single trainable object whose two parts are the model weights $heta$ and the executable harness $h$. The harness is not merely a prompt: it controls system instructions, tool schemas, context management, parsing, execution feedback, and termination. WHALE alternates two phases: a weight-update phase using online rejection-sampling fine-tuning under the current harness, and a harness-search phase using Meta-Harness under the updated model. Across SearchQA, mathematical reasoning, and Chess Puzzles, the paper reports best mean@8 improvements of 7.67 to 24.38 percentage points over single-component baselines and 4.15 to 13.00 points over a prompt-restricted Fast-Slow Training control.

This work sits between a method paper and a controlled empirical study. It is not primarily a leaderboard result; its central contribution is a simple recipe and a set of diagnostic experiments showing that the bottleneck moves depending on the domain. In SearchQA, harness search can match weight-only accuracy with far fewer rollouts, while in Math, harness search is almost useless until the model receives a small weight update. WHALE therefore belongs to the growing joint-adaptation lineage, but it advances it by moving the adaptable artifact from prompts to executable harness code and by studying the schedule of alternating updates under matched budgets.

Problem and Motivation

The problem is a shifting-bottleneck problem. In an agent with tools, the model decides what to write and what to call, but the harness decides what observations return, how tool schemas are formatted, what errors are surfaced, how many turns are allowed, and when an episode ends. Stronger weights cannot exploit evidence that a brittle retrieval harness never returns, while a better harness cannot compensate for a model that cannot synthesize the returned evidence. The paper makes this coupling explicit at the start of Section 1 and frames the agent as a pair $(heta,h)$ rather than a frozen model plus engineering scaffolding.

Prior joint adaptation work has usually optimized weights together with natural-language prompts. That is an important stepping stone, but it leaves the executable part of the system fixed: tool interfaces, context post-processing, parsing, retry logic, and termination policies remain hand-engineered variables. WHALE takes the next step by searching over executable harness programs. The paper motivates this extension by recent LLM-based optimizers that make direct source-code editing over agent harnesses feasible, including Meta-Harness.

The deeper difficulty is coordination. Harness search and online fine-tuning operate on different timescales. Harness search is a long proposal-evaluation-selection loop using many rollouts, while online fine-tuning interleaves rollout collection with gradient updates. If the two are run concurrently, every candidate harness is evaluated against a moving policy, and every gradient update is conditioned on a moving interface. If they are synchronized rigidly, the schedule becomes awkward. Alternation avoids both extremes by fixing one component while updating the other, but it introduces the critical design question: how long should each phase run before switching?

Core Recipe: Alternating Two Coupled Coordinates

1. The objective couples weights and executable code

The paper begins with a system-level reward objective:

Here $heta$ are the model parameters, $h$ is the executable harness, $\mathcal{P}$ is the task distribution, and $au$ is a trajectory containing model messages, tool calls, tool results, and environment transitions. The reward function $R$ maps trajectories to a scalar; in the experiments it is a sparse binary verifier that returns 1 only when the final task outcome is correct. The notation $J(heta,h)$ makes clear that success is not a property of the model alone: the same weights evaluated through different harnesses induce different trajectory distributions ${\pi }_{heta,h}$, and the same harness evaluated with different weights exposes different capabilities. If $J$ were separable, one could optimize each side independently, but Section 6.2 explicitly states that the objective is not separable. This is the formal reason that stagewise optimization is not automatically a good approximation.

2. Two conditional updates instantiate the two coordinates

For the weight-update side, WHALE uses online rejection-sampling fine-tuning, or RSFT:

In this expression, $het{a}_{\mathrm{old}}$ is the frozen rollout model, $h$ is fixed, ${\mathcal{S}}_s^{+}$ contains verifier-accepted pairs $(x,au)$ collected from prompt batch $B_s$, and $\mathcal{I}(au)$ denotes the positions of model-generated tokens while excluding user prompts and tool results. Compared with the original objective $J(heta,h)$, this estimator turns the weight-update phase into supervised learning only on the model's own successful trajectories. The token normalization prevents longer accepted trajectories from dominating simply because they contain more tokens. The design is intentionally simpler than policy-gradient training, and the paper cites reward-ranked self-training as the reason this family is chosen. Its failure mode is equally clear: if the current harness produces too few verifier-accepted trajectories, the update signal becomes sparse or empty; if the verifier is noisy, the model learns to satisfy the verifier rather than the latent task. This is most visible in SearchQA, where the verifier includes an LLM judge.

For the harness side, WHALE uses Meta-Harness:

This quantity evaluates a candidate harness $h$ under fixed weights $heta$ by averaging empirical verifier scores over a small harness-search dataset ${\mathcal{D}}_{\mathrm{harness}}$, which contains 256 examples in every domain. The paper says that one trajectory is sampled per candidate-example pair during harness search, so $\hat{\mathbb{E}}$ is a very low-variance-free empirical mean: it reflects the verifier outcome for a single rollout. Compared with the weight-update phase, this phase changes the interface rather than the parameters. It also maintains an archive of candidate harnesses and artifacts, including aggregate scores, per-example outcomes, and trajectory logs. The proposer can inspect these artifacts and refine or revert prior designs. The main tension is that harness search is cheap in wall-clock parameter updates but expensive in rollouts per candidate, and the reported rollout comparisons exclude proposer compute, as noted in Section 6.1.

3. Alternation and stopping decide whether co-adaptation works

The WHALE procedure alternates the two conditional updates:

This is the paper's essential methodological move. It is a black-box alternating update: any weight optimizer can plug into $\mathrm{ModelUpdate}$, and any harness-search algorithm can plug into $\mathrm{HarnessSearch}$. The paper instantiates it with RSFT and Meta-Harness, but the framework is broader than those components. Conceptually, the update is coordinate descent with one coordinate fixed at a time, and Section 1 cites the classic coordinate-descent view of alternating optimization. The reason for alternation is not merely convenience; it prevents credit-assignment confounding. A long weight-update phase optimizes behavior for the current harness $h_k$, and a long harness-search phase optimizes an interface for the current weights $het{a}_{k+1}$. If either phase is too long, it may reach a local optimum for the frozen counterpart and then fail when that counterpart changes.

The scheduling problem has two extremes. Too little evidence in a phase allows a noisy candidate or a random training fluctuation to determine the next state. Too much evidence causes over-optimization against a component that will soon change. The main WHALE configuration uses fixed budgets $(E,I)$, where $E$ is weight-update epochs per cycle and $I$ is harness-search iterations per cycle. The default comparison uses $(0.6,6)$. Adaptive WHALE replaces fixed budgets with a patience rule: after a minimum phase length, the weight phase stops when averaged training reward has not improved for a set number of steps, and the harness phase stops when the best archive score has not improved for a set number of iterations. This uses only training signals, not a validation set. The paper does not provide a convergence proof, which is consistent with the nature of the algorithm: both phases are stochastic, the objective is not separable, and the harness search is executed by an LLM proposer.

Experiments and Evidence

The evaluation uses Qwen3.5-2B for SearchQA and Math and Qwen3.5-4B for Chess Puzzles. All methods start from the same domain-specific initial model and harness. Weight-only trains for 4, 6, and 4 epochs respectively; harness-only runs 40, 60, and 40 harness-search iterations; WHALE alternates with $(E,I)=(0.6,6)$. FST is not used as an unmodified external method. The paper runs it as a prompt-restricted control using the same RSFT and Meta-Harness machinery but confines harness search to system and user prompts, isolating the effect of full executable-harness expressivity.

Table 2 reports the main test mean@8 accuracies:

The first observation is that the main gain is not uniform across domains. In SearchQA, harness-only already reaches 38.29 percent average, marginally above weight-only at 38.27 percent, while WHALE reaches 48.34 percent. This near tie supports the paper's claim that the harness can be the bottleneck in search QA: the base model is not hopeless, but the base harness does not retrieve, format, or terminate optimally. In Math, the pattern reverses. Harness-only collapses to 0.42 percent, whereas weight-only reaches 15.42 percent, and WHALE reaches 24.79 percent. This is a stronger contrast: prompt or tooling changes without changing the base model are nearly futile. In Chess Puzzles, WHALE reaches 29.83 percent versus 22.17 for weight-only and 25.68 for FST, again showing that full-harness search is not equivalent to prompt-only adaptation.

Figure 2 is the budgeted version of the same story. It tracks best-so-far test mean@8 accuracy as rollouts accumulate. Harness-only is held constant after 40, 60, and 40 iterations in the three domains, while WHALE interleaves short weight and harness phases. The figure supports the claim that joint updates can outperform either component under the same cumulative budget, and it foreshadows the later schedule analysis: small alternating steps dominate one-pass stagewise updates.

The behavior analysis in Section 6.1 is where the paper becomes especially useful. It decomposes the aggregate accuracy into capability-specific metrics. In SearchQA, Figure 3 tracks format compliance, retrieval success, and answer extraction. Weight-only peaks at 97.07 percent format compliance, while harness-only peaks at 99.98 percent, with fewer rollouts, because the harness can enforce answer formatting and final-answer behavior directly. For retrieval, the harness is more powerful: harness-only raises the fraction of trajectories retrieving at least one document containing the reference answer from 26.88 percent to 60.61 percent, and WHALE reaches 65.41 percent. Answer extraction reverses the balance: weight-only reaches 79.93 percent, whereas harness-only only reaches 57.28 percent. WHALE's trajectory here is non-monotone: the first harness-search phase improves extraction sharply, the second weight-update phase improves it further, but from cycle 3 onward later harness-search phases hold it back. That is a concrete illustration of why over-optimizing one coordinate against a later-changing counterpart can hurt.

Math offers the opposite diagnosis. The bottleneck is response truncation and final-answer extraction. Weight-only reduces the token-limit rate from 95.83 percent to 30.83 percent by learning shorter, more effective reasoning from verifier-accepted on-policy trajectories. Harness-only tries response caps, turn limits, and a final-answer recovery step, but cannot overcome the base model's behavior, so its format accuracy barely moves. WHALE changes this interaction: after the first cycle, its harness-search phase uses 4,608 rollouts to raise format accuracy from 0.83 percent to 4.38 percent, a gain of 3.54 percentage points. By comparison, harness-only uses 46,080 rollouts to gain only 0.63 percentage points, from 0.00 percent to 0.63 percent. This is the clearest evidence for the paper's "unlocking" claim: a short weight update can make the same harness search effective.

The schedule study is the other major evidence pillar. The paper compares five fixed $(E,I)$ schedules, stagewise optimization, adaptive WHALE, and baselines on SearchQA and Math. Table 3 reports the best test accuracy and rollouts consumed to reach it:

The stagewise comparison is decisive. Stagewise optimization first spends a long weight-only budget under $h_0$, then runs a long harness-only budget under the resulting model. Section 6.2.1 says this falls 5.32 points short of WHALE with $(0.6,6)$ in SearchQA and 9.16 points short in Math. More importantly, WHALE passes the final stagewise accuracy after only 29 percent of the stagewise rollouts in SearchQA and 49 percent in Math. This supports the coordinate-descent intuition: doing one large pass over each coordinate is dominated by smaller alternating passes when the coordinates interact.

The sweep also shows that the best fixed schedule is not at either extreme. In SearchQA, $(0.2,6)$ reaches 50.09 percent and $(0.6,6)$ reaches 48.34 percent. In Math, $(0.2,6)$ reaches 28.33 percent, clearly above $(0.6,6)$ at 24.79 percent. The noisy extreme appears in Math at $(0.2,2)$: with only two harness-search iterations per cycle, a chance-inflated candidate can be accepted, and the model then adapts to it. The paper states that the run destabilized and was stopped. The over-optimization extreme appears when per-cycle budgets are enlarged beyond $(0.2,6)$: accuracy falls to 48.34 percent and 45.93 percent along $(0.2,6)$ to $(0.6,6)$ to $(1.0,10)$ in SearchQA, and falls from 28.33 percent to 24.79 percent to 24.79 percent in Math.

Adaptive WHALE is the most interesting result. It removes fixed $(E,I)$ tuning by stopping each phase on patience over training signals. In SearchQA it reaches 52.82 percent, which is 4.48 points above the main $(0.6,6)$ schedule and 2.73 points above the best fixed sweep point, using 23 percent fewer rollouts than the $(0.6,6)$ best. In Math it reaches 26.46 percent, which is 1.67 points above $(0.6,6)$ but 1.87 points below the best fixed schedule. Section 6.2.3 reports that realized median budgets are 0.24 and 0.29 weight-update epochs in the two domains, with 7 harness-search iterations per cycle in both, while individual phases can extend to 1.16 epochs and 13 iterations. This means adaptive WHALE recovers a schedule close to the hand-tuned valley, but it does not universally beat hand-tuning.

The evidence quality is mixed in a useful way. The study is strong because it controls initialization, uses matched cumulative budgets, and separates full-harness WHALE from prompt-restricted FST. The behavior metrics in Figure 3 and the rollouts in Table 3 also avoid treating accuracy as a black box. However, several caveats matter. The rollout counts for harness-search efficiency exclude proposer compute, so the cost comparison is incomplete for wall-clock or monetary cost. The paper reports single best points in Table 3 and does not provide repeated-run variance in the provided text. Math has only 60 test problems in Table 1, so a large jump from 15.42 percent to 24.79 percent may still be noisy across seeds. Finally, adaptive WHALE is evaluated only on SearchQA and Math, not Chess Puzzles, and its Math result is below the best fixed schedule.

Insights, Limitations, and Outlook

WHALE's most durable insight is that agent training must decide when to stop each coordinate. A harness can be the more efficient adaptation channel in SearchQA, where retrieval behavior, answer formatting, and termination can be improved without changing the model. But the same harness search is almost useless in Math until the model learns to shorten responses and use the code interpreter effectively. A short harness search can therefore serve as a low-cost diagnostic: if it yields no progress after a bounded number of rollouts, the bottleneck is likely weight-limited. Conversely, if it yields a large jump, the harness is likely the dominant lever, and a small weight update may be enough to refresh the gains.

The practical recipe is simple. Use verifier-filtered online supervised updates for the weight phase, use a modular executable-harness optimizer for the harness phase, and interleave them in short cycles. Fixed budgets are easier to reproduce; adaptive patience is more automatic and outperforms the tuned fixed default in SearchQA while also improving over $(0.6,6)$ in Math. The paper's Figure 5, while conceptual, captures the main mechanism: the optimum lies between a noisy regime where phases stop too early and an over-optimized regime where each phase fits its frozen counterpart too tightly.

The limitations are equally concrete. WHALE is instantiated with one RSFT weight update and one Meta-Harness search method. The paper's black-box interface suggests broader applicability, but no other instantiation is reported. The search QA verifier depends on GPT-5.4-mini judging the extracted answer under a rubric, which means the training signal can reward judge-accepted behavior rather than unambiguously correct factual retrieval. The harness-search budget uses only 256 examples, so candidate selection can be noisy. The FST control is a reimplementation rather than a direct comparison to an original system under the same exact prompt optimizer, although the authors are explicit that the purpose is to isolate expressivity of the harness search space. Scaling beyond 2B and 4B models remains open.

Future work should address the missing pieces of the current evaluation. One direction is to include proposer-compute accounting so that harness-search efficiency claims become complete. Another is to evaluate on coding or computer-use agents, where harness edits change sandbox rules, persistent memory, test execution, and error recovery rather than only retrieval formatting. A third is to combine adaptive WHALE with denser or model-based verifiers, because sparse binary rewards can make both RSFT and harness selection vulnerable to near-zero accepted sets. WHALE's contribution is that it gives a tractable starting point: treat the model and harness as one system, alternate small updates, and let the training signals choose the schedule.