Can an open agentic retriever handle hard multi-step search?

T-Search returns ranked evidence, not answers

T-Search is an open-weight agentic retriever built for hard multi-step search. Given a question and a search tool over a fixed corpus, it runs a bounded multi-round search and returns a ranked list of evidence chunks, each with a short justification. It never writes the final answer; that is left to a downstream model of the user's choice.

This split is the design's central claim. Because the retriever only reaches documents through the search tool, the backend can be swapped without retraining, and because it returns chunk identifiers, its quality can be scored directly as recall of gold evidence rather than through an answer judge. One retriever can therefore serve many generators, and retrieval quality is no longer conflated with generation quality.

Recall@10 measures selection under a ten-item cap

The headline metric is Recall@10 on the final ranked list. For each question, the returned list contains at most ten items; each identifier is mapped to the scoring unit of that benchmark, and recall is the size of the intersection with the reference evidence set divided by the size of that reference set. The ten-item limit is applied before mapping and deduplication, so the scored set can contain fewer than ten documents.

The scoring unit differs by benchmark, and this matters for reading the numbers. BrowseComp-Plus is scored at document level in both languages, and multiple chunks from the same reference document earn credit only once. TRuST maps a docid_chunkid identifier back to its parent document before deduplication. The SealQA adaptation and SynthComp preserve the evidence-chunk identifier. For SynthComp, the denominator includes all evidence chunks, not only the answer-bearing gold subset, because the remaining supports establish constraints and connections that identify the answer.

A separate trajectory recall metric uses all unique search-backend results without a ten-item cutoff. It measures accumulated search coverage, while final-set Recall@10 additionally measures source selection under the ten-item limit. The two should not be read as the same quantity.

SFT and RL each add roughly seven points

T-Search is built on Qwen3.6-35B-A3B, a mixture-of-experts model with 35B total and about 3B active parameters. Training uses adversarially filtered synthetic search tasks, round-sliced supervised fine-tuning, and then GSPO on a recall reward.

The stage comparison shows a consistent ladder. Averaged over the seven benchmarks, the base model scores 41.5 Recall@10, the SFT model 49.0, and the released model after SFT plus RL 56.0. That is a 7.5-point gain from SFT and a further 7.0 points from RL. Every individual benchmark follows the same ordering: the SFT model beats the base, and the released model beats the SFT model.

Two details qualify this comparison. Each trained model already includes the merge of its English and Russian branches, so the table does not separate the effect of training from the effect of merging. And the reward is a single scalar per trajectory, the recall of the reference support set in the finalized ranking, computed from chunk identifiers without a judge model. There is no separate reward for saving evidence and no per-round shaping. The authors report that recall was preferred over precision-based objectives because precision-based objectives led to premature finalization with incomplete evidence in preliminary runs.

Bounded rounds keep the working window short

The harness runs a ReAct-style loop with short, bounded rounds that carry forward only explicitly saved evidence, so tool output does not accumulate and degrade the model's judgment. The default per-round budget is 32,768 tokens and the default round cap is five.

Several guardrails define the loop. At 75% context utilization, further search calls are locked, leaving only continuation or finalization. A save-and-advance transition requires at least five searches and at least one saved chunk with an explanation, though a task solved earlier can finalize without making five queries. Saving is disabled in the last allowed round. The harness rejects repeated queries that differ only in case or whitespace, filters results already seen in the current round, and filters chunks already carried in context. Invalid tool calls receive error feedback, and during SFT such invalid actions are masked from the loss while the continuation is kept.

The authors treat the harness as part of the model rather than as neutral scaffolding. The rounds, the three-tool contract, and the guardrails are what the policy learned to operate inside, so quality can shift in a different loop. Their recommendation is to run the released harness and validate on your own index before relying on the reported numbers.

Three fused rollouts add 5.3 points

With one rollout, T-Search averages 56.0 Recall@10 across the seven benchmarks. With three rollouts fused by reciprocal rank fusion, the average rises to 61.3, a gain of 5.3 points. For the fused case, each of the three ranked lists contains at most ten items, the fused ranking is truncated to ten, and only then is the benchmark-specific mapping and deduplication applied.

This is a scenario split rather than a free improvement: the higher number comes from spending three rollouts per question instead of one. The paper reports both figures, and the single-rollout number is the one that matches the stage comparison against the base and SFT models.

Backend choice changes the score by up to 11 points

Because the agent reaches documents only through the search tool, the backend can be substituted while the T-Search agent stays fixed. Under that substitution with one rollout, Qwen3-Embedding-8B with Qwen3.6-27B as an LLM reranker gives the highest macro-average at 62.87. Qwen3-Embedding-8B without reranking averages 55.96, Jina v5 text-small 54.32, BM25 51.14, and Qwen3-Embedding-0.6B 49.38.

The spread is not uniform across benchmarks. BM25 is the strongest of the tested backends on SynthComp and TRuST, while its BrowseComp-Plus scores are lower. The authors note that these comparisons measure sensitivity to backend choice and do not isolate the contribution of the multi-retriever SFT subset.

The practical reading is that the reported 56.0 average is tied to a specific retrieval environment. Changing the embedding model or adding reranking moves the number, and the best backend depends on which benchmark matters for a given deployment.

The comparison standardizes retrieval but not budgets

The main comparison places T-Search against adapted released search-agent harnesses over the same fixed indices, using Qwen3-Embedding-8B without reranking or public-web access, with each search-result snippet capped at 512 tokens. The authors state that T-Search outperforms larger open models on average over the seven fixed-index benchmarks.

The comparison scope is explicitly bounded. Agents differ in interaction budgets, context management, auxiliary-model use, and the number of sources included in their reported metric. The Alibaba adapters derive from released DeepResearch code, Harness-1 and Search-R1 use their own released revisions, and the experiments evaluate those agents in this retrieval environment rather than reproducing their original benchmark settings. T-Search itself uses no auxiliary language model, while the four Alibaba agents share a Qwen2.5-72B-Instruct visit summarizer.

Two further limits bound the claim. T-Search cannot be deployed or evaluated as a standalone question-answering system, since it returns evidence and leaves generation downstream, and on general-purpose benchmarks the authors did not observe noticeable differences from the base model. The model is also trained and evaluated only in English and Russian, so behavior in other languages is untested.

Swap the backend, keep your generator

The paper's own framing of the practical payoff is narrow and specific: because the agent reaches documents only through a tool, practitioners can swap the backend, point the harness at their own index, and keep the generator they already run. The released artifacts include the model with FP8 and NVFP4 quantizations, the harness, a live demo, and three benchmarks, including TRuST, described as the first native-Russian hard-search benchmark.

The decision this evidence supports is about where to spend effort. If the bottleneck is finding the right evidence chunks in a fixed corpus, the reported 56.0 single-rollout average and the 61.3 three-rollout average define what this retriever achieves under a five-round cap, and the backend-substitution table shows how much that number moves with the embedding model. If the bottleneck is answer quality, this paper does not address it, because generation is deliberately out of scope.

The most useful comparison for a reader is therefore not T-Search against a single baseline but the pair of splits the paper itself reports: one rollout versus three fused rollouts, and the choice of retrieval backend. Both change the headline number, and both are decisions a practitioner makes rather than properties of the model.

Sources used in this answer