How can dexterous manipulation be benchmarked with dynamic phenomena in laboratories?

A benchmark built around laboratory procedures, not object moving

AutoBio+ is a simulation benchmark and framework for dexterous manipulation in biological laboratories. Its 14 tasks are grouped by four capability categories: functional tool use, precision manipulation, language-conditioned visual grounding, and instrument operation. The point of the grouping is that a policy must do more than grasp and relocate an object: it has to use a pipette or tweezers for their intended function, align fingertips on a tube cap, find a slot named in an instruction, or actuate a control on an instrument.

The suite spans four robot embodiments, so the same benchmark covers different hands and arms rather than a single gripper. Task randomization is specified per task: for example, the tip slot in Attach Pipette Tip is sampled from a 6×4 tip-box grid, and the cryovial slot in Aspirate with Pipette is sampled from a 5×12 rack grid, while robot reset ranges are given in radians for each embodiment. These details matter because they define what counts as the same task across trials.

Flames, vapor and bubbles change what the camera sees

The paper's distinctive addition is a set of controllable dynamic experimental phenomena: flames, vapor and bubbles. They are rendered by a particle system whose particles carry position, color, opacity and size, and are updated by a drift term plus a small random perturbation rather than by full physics simulation. The authors describe this as favoring perceptual fidelity while staying lightweight and parallelizable.

Each phenomenon is tied to a laboratory procedure. The flame is made of upward-moving particles whose color is modulated by the rotational positions of the Bunsen burner's two rings, so the appearance changes with burner adjustment. Vapor is modeled as expanding particle clouds with randomized drift and dispersion, and bubbles as rising particles with vortex-like trajectories. The benchmark therefore tests policies under observations that evolve during a procedure, not only under static scenes.

Real laboratory backgrounds sit behind simulated objects

To make scenes look like real laboratories without giving up physical interaction, AutoBio+ uses a hybrid rendering pipeline. Interactive foreground objects are still rendered by the simulator, while the background comes from 3D Gaussian splatting assets reconstructed from six real laboratory scenes captured with a lightweight spatial scanner. The final image combines the simulator's object-level foreground mask with the Gaussian-rendered background.

This design is a deliberate compromise: the authors state that it retains physically controllable simulated objects while adding realistic visual context, rather than replacing the simulator renderer entirely. The cost is measured. On a benchmark of 339 frames from two cameras at 640×480, plain MuJoCo Warp rendering took 13.95 s, Gaussian-augmented rendering took 15.57 s, and the Blender rendering used by the earlier AutoBio took 1092.02 s. The authors report this as roughly 70.14× faster than AutoBio with about 11.61% overhead over plain MuJoCo Warp.

Parallel simulation makes large-scale evaluation practical

AutoBio+ is built on mjlab and adds a parallel simulation framework. Its central component is a Parallel Finite-State Machine (PFSM), which the authors use for demonstration generation, together with batched rendering and decoupled trajectory replay for model-independent closed-loop policy evaluation. In plain terms, many simulated environments run at once, and recorded trajectories can be replayed without tying evaluation to one particular policy implementation.

The rendering pipeline supports batched Gaussian rendering on the GPU, and the particle effects are updated independently per particle, so they do not require global synchronization. The paper also extends interactive textures by predefining a set of textures in the scene description and indexing them at runtime, because Warp does not support arbitrary texture changes during simulation. These are engineering choices aimed at throughput, and they are what allow the benchmark to be used for both data generation and repeated policy comparison.

No policy solves the suite, and the gaps are task-specific

The main evaluation compares four policies — Diffusion Policy, π0.5, GR00T-N1.7 and Fast-W AM — on 12 representative tasks chosen from the 14, stratified across the four capability categories and including all four embodiments. All models are trained on the same task-specific dataset and evaluated with identical domain randomization and task-specific termination criteria. Results are mean success rate with standard error over 3 seeds, 100 rollouts per seed, 300 rollouts total; all tasks use binary rewards except Operate Mixer Panel, which uses a progress reward.

The spread is large. On Close Thermal Cycler Lid, success ranges from 5.3% for π0.5 to 99.7% for Fast-W AM, while on Close Centrifuge Lid GR00T-N1.7 reaches 100% and Diffusion Policy 30.0%. On Ignite Bunsen Burner, π0.5 reaches 77.7% and Fast-W AM 23.3%. At the other end, Tighten Tube Cap is 0.0% for three policies and 0.7% for π0.5, and several language-conditioned grounding tasks stay in low single digits. The authors read this as exposing distinct strengths and weaknesses rather than a single ranking.

Contact-rich manipulation and language grounding remain bottlenecks

The paper's discussion separates two failure patterns. First, language-conditioned visual grounding tasks remain generally low and highly variable, and no task in that category is solved reliably; some models reach 0% on certain tasks. These tasks require grounding a language-specified target, such as a tube slot given by row and column, in the current observation. The authors note that Fast-W AM does not consistently beat the vision-language-action baselines here, and suggest the limitation may lie in the language-conditioned front end rather than in predicting future visual states.

Second, contact-rich manipulation is a prominent failure mode. Policies do worse on manipulating tweezers, tightening tube caps, attaching pipette tips, loading centrifuges and transferring tubes, and better on nonprehensile interactions such as closing centrifuge lids and opening thermal cycler lids. The authors also observe that dexterous-hand pretraining helps on relatively simple or tolerance-rich interactions — GR00T-N1.7's 100% on Close Centrifuge Lid is their example — but does not carry over to tasks needing precise alignment and continuous finger adjustment.

Gaussian backgrounds lift a few-shot real-robot score

The real-world validation uses a UR5e arm with an XHand and two RealSense cameras, a wrist-mounted D405 and a third-person D435. Each training set contains 10 real-robot demonstrations and 90 simulated demonstrations, and the two conditions differ only in the simulated visual background: Gaussian-splatting backgrounds versus the standard mjlab background. Evaluation uses a progressive score on the same real-robot task: 3 points for grasping the tube, 5 for lifting it, 8 for initiating insertion, and 10 for fully inserting it into the rack. The tube is placed at each of ten rack positions and tested twice, giving 20 trials per condition.

The Gaussian-background condition reaches a normalized mean score of 66%, against 17% without Gaussian backgrounds. The authors interpret this gap as evidence that realistic background appearance matters particularly in the few-shot regime, where 10 real demonstrations cannot by themselves cover the task's visual variation. This is a comparison of two rendering conditions on one real-robot task, not a general claim that simulated data replaces real data.

Scope limits sit in task coverage and scene fidelity

The authors state two limitations. Task and asset scalability: AutoBio+ does not yet provide a framework for evaluating the full scope of biological laboratory automation, and future work could synthesize assets and tasks generatively to reduce manual effort. Rendering and dynamic-scene fidelity: the Gaussian-background compositing plus particle system may not fully reproduce high-fidelity liquid behavior or the full range of scene interactions, and the authors point to 4D Gaussian representations or dynamic neural fields as possible directions.

These limits qualify how the results should be read. The dynamic phenomena are visual effects decoupled from scene physics, so they test perception under evolving appearance rather than the physics of liquids or flames. And the policy comparison covers 12 of the 14 tasks, so the reported strengths and weaknesses describe that stratified selection, not every task in the suite.

Grounding and contact control matter more than world modeling alone

The most decision-relevant split in this paper is between task types rather than between policies. Where a task mainly requires operating an instrument or a nonprehensile interaction, some policies already reach high success — 100% on Close Centrifuge Lid and 99.7% on Close Thermal Cycler Lid. Where a task requires grounding a language-specified target or precise contact-rich finger coordination, success collapses toward single digits across the board.

For readers deciding what to build or measure next, that split sets the priority: the benchmark's own evidence says the remaining gap is concentrated in language-conditioned target identification and contact-aware fine control, not in the ability to predict future visual states. The Gaussian-background result adds a second, narrower threshold: in a few-shot setting with 10 real demonstrations, the choice of simulated background alone moved the real-robot score from 17% to 66%, so background fidelity is a variable worth controlling when simulated data is mixed with real data.

Sources used in this answer