What has changed: from open-loop to closed-loop claims
Previously, embodied AI systems were largely open-loop: they followed fixed skills during execution and only reflected after an episode ended. That meant failures during a task went uncorrected in real time. Recent work has shifted to closed-loop harnesses that can diagnose and repair failures while the task is still running. For example, Zetta uses three timescale-separated loops to govern actions at high frequency, propose critic-recovery plans at rollout level, and update skills only after validation [2]. This is a fundamental change because it means the system can adapt mid-task, not just after the fact.
Another key shift is the introduction of explicit failure diagnosis and repair mechanisms. Onto-EV-WM adds an ontology-grounded layer that records exactly which task predicate was unmet, what correction route was available, and whether the correction was accepted [1]. Similarly, Thea introduces 'Evaluation as Exit Codes' to detect when an action should terminate, judge success, and diagnose causes of failure [3]. Regulators should look for these kinds of explicit, auditable failure-handling components, not just overall success rates.
What should regulators require in evidence?
First, they should require standardized, reproducible benchmarks. The papers show that performance varies dramatically with the task suite: Onto-EV-WM achieved 85% overall on a 10,030-task registry, but only 65.98% on the LIBERO-10 subset [1]. A claim of '94% success' means little without specifying the exact tasks, simulator, and evaluation protocol. Regulators should mandate that claims be tied to a defined benchmark with clear success criteria.
Second, they should require evidence of failure diagnosis and repair, not just final success. The papers emphasize that closed-loop harnesses must be able to identify why a task failed and apply a correction. For instance, Onto-EV-WM retains missing predicates and their arguments, enabling targeted repair [1]. Regulators should ask for logs or traces that show the system detected a failure, selected a correction route, and verified the correction—not just that the task eventually succeeded.
Third, they should require auditable, traceable measurement. MMRHP provides a structured three-phase testing process aligned with the Safety of the Intended Functionality (SOTIF) standard, with a unified spatiotemporal measurement core to ensure consistent and traceable quantification of physical motion and system timing [4]. Regulators should look for similar rigor: clear definitions of what constitutes success, how timing is measured, and how physical motion is tracked. Without this, claims are not scientifically reproducible.
What are the limits and caveats?
The evidence is largely from simulation, not real robots. Onto-EV-WM explicitly states that real-robot recovery is not evaluated [1]. Zetta reports results on LIBERO-Pro and RoboCasa, which are simulation benchmarks [2]. Regulators should treat simulation results as promising but not sufficient for real-world deployment, especially in safety-critical domains like autonomous driving, where MMRHP found a 'performance cliff' at an injected latency of 40 ms [4].
Another caveat is that some reported success rates are not directly comparable across papers due to different task suites and evaluation protocols. For example, Zetta reports 90.8% on LIBERO-Pro [2], while Onto-EV-WM reports 91.39% on LIBERO-Goal [1]—but these are different benchmarks, so one cannot conclude which system is better. Regulators should demand head-to-head comparisons on identical benchmarks before making decisions.
Finally, the papers do not provide evidence on long-term reliability, safety, or ethical implications. The urban automation framework [5] raises concerns about institutional accountability and democratic legitimacy, which are not addressed in the technical papers. Regulators should consider these broader implications, especially for systems that will operate in public spaces.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2025 to 2026, 5 from 2024 or later — selected as the most relevant from 5 studies that passed quality screening, drawn from 72 papers retrieved from a database of over 500 million.
Sources used in this answer
Ontology-Grounded World Models for Failure Diagnosis and Closed-Loop Repair in Physical AI Systems
Onto-EV-WM, an ontology-grounded diagnosis and verification-gated correction interface, achieved 85% success on a 10,030-task registry, with 65.98% on LIBERO-10 and 91.39% on LIBERO-Goal, but real-robot recovery was not evaluated.
Zetta $\zeta$: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence
Zetta, a closed-loop embodied harness with three timescale-separated loops, achieved 90.8% success on LIBERO-Pro and 93.6% on RoboCasa with an 11.1x inference speedup, and skills transferred zero-shot.
Towards the Harness of Embodied Agents
Thea, a harness for embodied agents, introduces Scene Graph as Context and Evaluation as Exit Codes to close the loop between agent and physical world, enabling long-horizon tasks in real environments.
MMRHP: A Miniature Mixed-Reality HIL Platform for Auditable Closed-Loop Evaluation
MMRHP, a miniature mixed-reality hardware-in-the-loop platform, provides a structured three-phase testing process aligned with SOTIF, achieving 10.27 mm RMSE spatial accuracy and identifying a performance cliff at 40 ms injected latency in an Autoware case study.
From Optimisation to Closed-Loop Urban Automation: A Conceptual Framework for Spatial Intelligence and Physical AI in AI–IoT-Enabled Smart Cities
A conceptual framework for closed-loop urban automation differentiates it from optimisation-centric models and proposes six propositions, highlighting the need for institutional accountability and democratic legitimacy in AI-IoT-enabled smart cities.