Why start in simulation? Because the model must prove it can handle the full loop of perceive-act-verify.
Execution-centric VLMs don't just answer questions—they drive actions that change the world, so testing them requires a closed loop: the model perceives a scene, decides an action, the action changes the scene, and the model must re-perceive and verify. Simulation is the only safe place to run this loop at scale. The Capek 0.5 work explicitly evaluated its model in simulated embodied environments and showed it transferred to closed-loop task execution, meaning the model could actually complete tasks, not just answer benchmarks [6]. Similarly, a surgical robot learning platform was built specifically to allow human-in-the-loop simulation, and using human demonstrations in that simulated environment improved reinforcement learning efficiency [5].
The key point is that simulation lets you test the iterative nature of execution—each action reshapes the scene, so the model must continually update its perception and reasoning [6]. Without this closed-loop test, you might have a model that scores well on static benchmarks but fails when its own actions change the world. So the first testing layer is: run the model in a high-fidelity simulator, let it act, and check whether it can complete tasks from start to finish, not just answer isolated questions.
Break the model into its core capabilities and test each one—then test the whole.
Execution-centric VLMs need several distinct skills: spatial reasoning (where things are), temporal understanding (what happened before and what will happen next), action guidance (what to do next), and state verification (did the action actually work). Capek 0.5 was built around exactly this taxonomy, and its evaluation included a controlled study of capability retention from specialists to the unified model—meaning they tested each capability in isolation and then checked that the combined model still retained all four [6]. This is a crucial testing approach: if you only test the final model, you might miss that it lost spatial reasoning even though it still follows instructions.
The same paper introduced a new benchmark, Capek-StateBench, specifically for state verification—the ability to check whether the scene matches what you expected after an action [6]. This is often the weakest link in embodied systems, because a model might plan a correct action but fail to notice that the action didn't have the intended effect. So your testing protocol should include: (1) separate benchmarks for each capability, (2) a combined benchmark that requires all capabilities simultaneously, and (3) a state-verification benchmark to catch the 'did it actually work?' failure mode.
Before humans are involved, you must test how the model behaves under adversarial and unexpected inputs.
Safety is not just about avoiding crashes—it's about robustness to malicious or accidental input corruption. A 2026 survey on embodied intelligence security highlights that adversarial attacks on visual or textual inputs can cause incorrect interpretations and hazardous actions in VLM-based systems [3]. For example, a slightly altered image or a misleading instruction could make the model think a person is not there, or that a tool is safe to grab when it isn't. The survey emphasizes that integrating VLMs into physical agents amplifies these risks because errors have immediate physical consequences [3].
So your testing must include adversarial robustness checks: feed the model corrupted or adversarial images and text, and see if it still makes safe decisions. The survey also notes that VLMs can be used defensively to enhance reliability, so you might test whether the model can detect when its own perception is unreliable [3]. This is a distinct testing layer from functional capability—it's about ensuring the model fails safely when things go wrong.
Finally, bring humans into the loop—but in a controlled, simulated or supervised setting first.
The ultimate test before real-world human interaction is human-in-the-loop evaluation, where humans either demonstrate tasks or supervise the model's actions. The surgical robot study is a prime example: they built an interactive simulation platform that allowed human input through a physical device, and used human demonstrations to improve policy learning [5]. This shows that human-in-the-loop testing isn't just about safety—it can also improve the model's learning efficiency.
In human-robot collaborative assembly, VLMs are used to decompose tasks between humans and robots, and to generate guidance for human workers [1][4]. Testing such systems requires simulating the full collaboration: the model must understand human actions, communicate instructions, and adapt to human behavior. The 2025 study on VLM-enhanced embodied intelligence for human-robot collaborative assembly used a digital twin—a virtual replica of the physical scenario—to simulate and deduce outcomes, which helped mitigate collision accidents [1]. So before letting the model interact with a real person, run it in a digital twin or interactive simulation with a human in the loop, and measure not just task success but also safety metrics like collision avoidance.
One caveat: the evidence here is largely from research prototypes, not large-scale deployment. The 2025 paper on VLA models lists 'coordination with humans' as one of ten open challenges, indicating that human-robot coordination is still an unsolved problem [2]. So while human-in-the-loop simulation is a necessary step, it's not sufficient—you should also plan for gradual real-world deployment with close supervision.
About These Sources
This answer is built on 6 studies (4 peer-reviewed, 2 preprints) — published from 2023 to 2026, 5 from 2024 or later, 3 in Q1 journals, collectively cited 60 times — selected as the most relevant from 8 studies that passed quality screening, drawn from 49 papers retrieved from a database of over 500 million.
Sources used in this answer
Vision language model-enhanced embodied intelligence for digital twin-assisted human-robot collaborative assembly
Proposed a VLM-enhanced embodied intelligence framework for human-robot collaborative assembly, using a digital twin for simulation and deduction; comparative experiments in a real-world HRC scenario showed competitive advantages over traditional methods like MA-A2C, DQN, and GA.
10 Open Challenges Steering the Future of Vision-Language-Action Models
Identified 10 open challenges for vision-language-action models, including safety, coordination with humans, and evaluation, highlighting that these are unresolved issues.
Embodied Intelligence Security with Vision-language Models: A Survey
Surveyed security risks in VLM-based embodied intelligence, noting that adversarial attacks on visual or textual inputs can lead to hazardous actions, and that physical interaction amplifies safety concerns.
Vision language model-enhanced embodied intelligence for AR-assisted HRC assembly: Multimodal cognition, task reasoning, and autonomous execution
Proposed an end-to-end VLM-enhanced framework for HRC assembly with four modules (perception, reasoning, AR guidance, and code generation); experiments showed superiority over traditional models and LLMs.
Human-in-the-Loop Embodied Intelligence With Interactive Simulation Environment for Surgical Robot Learning
Developed an interactive simulation platform for surgical robot learning that supports human-in-the-loop interaction; human demonstrations improved reinforcement learning efficiency, and five new training tasks were released.
Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence
Presented Capek 0.5, an execution-centric VLM with four capability families (spatial, temporal, action, verification); evaluation included a new state-verification benchmark and closed-loop simulation, showing improved performance on most benchmarks and retention of all capabilities.