How does OneSearch-VL unify multimodal deep research for images and video?

OneSearch-VL trains a single agent on a shared evidence graph for image, multi-image and video research, gaining 20.2 and 17.6 points.

Direct answer

OneSearch-VL is a single 8B agent trained to do deep research over single images, multi-image collections and video by sharing one structure: the Visually Grounded Evidence Graph, which links localized visual anchors to entities, source-supported facts and the operations that compose an answer. With tool access, it beats Qwen3-VL-8B by 20.2 and 17.6 percentage points on the paper's new multi-image and video benchmarks, and also gains on 7 image benchmarks and VideoDR.

1sources cited

This article was generated with WisPaper-powered search and paper analysis.

One workflow spans three visual input types

The paper starts from an observation rather than a new benchmark score: single-image, multi-image and video research need different visual operations, but they follow the same workflow. The agent first grounds a visual clue, then retrieves external knowledge, then composes retrieved facts into an answer. Video adds temporal localization and frame selection; multi-image adds the need to relate entities spread across separate images.

That shared workflow is the justification for unified training. Instead of building separate policies for each input type, the authors train one agent on data spanning all three, so that grounding, searching and fact composition are learned as one skill family rather than three disconnected ones. The claim being tested is therefore narrow and checkable: whether one policy can jointly learn deep research over images and videos.

The evidence graph keeps provenance from pixel to answer

The central object is the Visually Grounded Evidence Graph (VGEG), a task-level reference structure. It links a localized visual anchor — a region, an image, or a video frame — to a real-world entity, records multi-hop relational paths and the web facts that support them, and specifies the operations that turn those facts into an answer.

The point is dependency, not storage. Prior structures such as perception–knowledge chains, factual rubrics and evidence graphs each capture part of the picture, but the paper argues they do not jointly preserve the chain from a localized visual anchor through entity relations and source-supported facts to answer composition. Video pipelines in particular tend to use extracted entities or key-frame crops only as retrieval or question-generation seeds, without retaining end-to-end provenance. VGEG is designed so that the same annotation can serve four jobs: building questions, supervising process, rewarding trajectories, and evaluating operations.

A data engine builds and verifies each task

The VGEG is produced by a staged pipeline. For video, the source pool starts at 2.5M candidate records and is filtered by upload date (on or after January 1, 2023), metadata scoring, category balance and exclusion, an LLM snippet filter, and duration balancing, ending at 70,781 records. Frames are sampled at 2 FPS, grouped into ten-frame clips, captioned, deduplicated by image and text embedding similarity, and aggregated into events, giving a hierarchy of video, event, key frame and localized object.

Localized objects are then routed to reverse image search or OCR-plus-text-search, verified against retrieved pages, and expanded into a web entity graph whose facts each carry a head entity, relation, tail entity or value, supporting URL and quotation. Questions are generated from this graph and then checked: a QA verifier re-answers each question using only the gathered evidence and requires semantic agreement with the proposed answer, and a question-only probe rejects items the model can solve without the visual input. Explicit entity names are rewritten into descriptions so the question starts from visible targets.

The same machinery yields the training sets: about 110K samples for supervised fine-tuning and a 10K reinforcement-learning set, the latter kept only for tasks where eight rollouts by Qwen3-VL-8B produced between zero and eight correct answers.

The reward scores traceability and grounding separately

The Evidence-aware Visual-Grounded Rubric reward (EVGR) turns VGEG annotations into a process reward with two dimensions. Evidence traceability asks whether tool observations actually establish the answer-core entities and required fact hops, and whether later reasoning follows those observations without unsupported substitutions. Visual grounding asks whether the correct visible entities, regions or frames were identified and used to drive subsequent retrieval.

A judge model scores each dimension on anchored levels from 0.00 to 1.00, where 1.00 means all required fact hops are directly supported and all required visual anchors are correctly grounded and used, and 0.00 means no supporting evidence or no grounding is established. The rubric explicitly avoids rewarding verbosity, fluent but unsupported reasoning, ineffective tool calls, or final-answer matching alone. In the composite reward, answer correctness, query quality and EVGR carry weights of 0.6, 0.2 and 0.2, gated by format validity.

One asymmetry matters for reading the grounding score: for single-image and multi-image inputs the source images are already visible to the policy, so no extra visual tool call is required and the criterion checks that the right images and objects were used; for video, selected frames are compared against reference moments and must supply the visual identity that later searches rely on.

Gains concentrate on operation-level benchmarks

The headline comparison is against Qwen3-VL-8B with tool access. OneSearch-VL-8B improves by 20.2 percentage points on OneSearch-MI-Bench and 17.6 percentage points on OneSearch-Video-Bench, the two benchmarks the authors build by organizing questions according to the research operations encoded in their VGEGs. The paper also reports substantial gains across 7 image benchmarks and VideoDR.

These numbers should not be read as one uniform improvement. The two new benchmarks are constructed by the same pipeline that produces the training data and are organized by operation, so they measure the specific skills the VGEG supervision targets; the 7 image benchmarks and VideoDR are separate evaluations reported as gains without a single combined figure in the evidence available here. The comparison baseline is Qwen3-VL-8B with tool access, not a tool-free model, so the margin reflects the training recipe rather than tool availability alone.

Cross-input supervision and EVGR both contribute

The paper reports ablations showing two things: supervision across visual input types is complementary, and EVGR adds benefit beyond the outcome and query rewards. The complementarity claim is the more consequential one for the unification question, because it implies that training on multi-image and video tasks does not merely add coverage but helps the shared workflow.

The evidence available here states these ablation conclusions without the per-condition numbers, so the size of each contribution cannot be compared from this packet. What can be said is directional: the authors attribute part of the unified agent's performance to cross-type data and part to the process reward, and they treat the two as separate design choices rather than one bundled recipe.

Tool dependence limits exact reproducibility

The agent depends on external search tools such as TextSearch and ImageSearch and on changing webpages, which the authors note can affect evidence availability and exact reproducibility. Automated VGEG construction and model-based rewards can also introduce annotation errors or judging biases, and multi-turn interaction adds inference and tool cost.

These limits qualify the benchmark numbers rather than the conceptual claim. A 20.2 or 17.6 point margin was measured in a specific tool environment against a specific baseline; reproducing it requires the same retrieval conditions, and the reward's reliance on a judge model means the process signal inherits that judge's biases. The authors propose preserving retrieval snapshots, testing robustness to tool failures, and using human calibration or open multimodal process judges as future work.

Unification is the claim worth testing

The paper's distinctive move is not a new visual encoder or a larger model but a shared reference structure that makes one training signal serve question construction, trajectory filtering, process reward and operation-level evaluation. If a reader wants to judge whether unified multimodal deep research is real, the relevant comparison is between one policy trained across input types and per-type policies, and the relevant evidence is the cross-type ablation plus the operation-organized benchmarks.

The practical priority follows from that: the 20.2 and 17.6 point gains on OneSearch-MI-Bench and OneSearch-Video-Bench are the results to scrutinize first, because they are measured on tasks built from the same VGEG operations the agent was trained to perform. The 7 image benchmarks and VideoDR indicate the gains are not confined to the new benchmarks, but they are reported as aggregate gains rather than as a controlled test of transfer.

Sources used in this answer

1

OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video

OneSearch-VL-8B, trained on VGEG-derived data with an evidence-aware process reward, improves over Qwen3-VL-8B with tool access by 20.2 and 17.6 percentage points on OneSearch-MI-Bench and OneSearch-Video-Bench, with further gains on 7 image benchmarks and VideoDR.