What actually makes people trust an AI system?
Trust in AI isn't just about technical accuracy—it's about the system's perceived ability, integrity, and benevolence. A 2021 interview study with experts from various companies found that access to knowledge, transparency, explainability, certification, and self-imposed standards are the key factors that increase overall trust in AI [1]. For world model recovery, this means users need to see how the system works, why it makes decisions, and that it has been independently verified—not just that it performs well in tests.
Why certifying the recovery system—not the whole model—could be the key
A 2024 study on reinforcement learning systems suggests a practical path: only the fault recovery system needs to be certified, not the entire AI model [2]. This is a game-changer because world models are complex and hard to fully verify, but a recovery system that detects when the model is out-of-distribution (OOD) and safely intervenes can be simpler and more testable. By focusing certification on this safety-critical component, users get a clear, auditable guarantee that the system will fail safely—which is exactly what builds trust.
The catch: over-trust can create dangerous illusions
A 2026 survey on world-model-based embodied AI warns that world models can serve as safety shields, but when compromised or over-trusted, they generate 'predictive safety illusions' [3]. This means that if users trust the recovery system too blindly, they might ignore signs that the model is failing. The survey outlines evaluation protocols for safety failures and defenses like robust grounding and uncertainty-aware prediction, emphasizing that trust must be continuously earned through rigorous testing—not assumed once and forgotten.
About These Sources
This answer is built on 3 studies (2 peer-reviewed, 1 preprint) — published from 2021 to 2026, 2 from 2024 or later, 1 in Q1 journals, collectively cited 278 times — selected as the most relevant from 3 studies that passed quality screening, drawn from 48 papers retrieved from a database of over 500 million.
Sources used in this answer
Can we trust AI? An empirical investigation of trust requirements and guide to successful AI adoption
A 2021 interview study with experts from various companies identified transparency, explainability, certification, and self-imposed standards as key factors that increase trust in AI, highlighting fundamental differences from traditional technologies.
Can you trust your Agent? The effect of out-of-distribution detection on the safety of reinforcement learning systems
A 2024 study on reinforcement learning systems argues that for safety-critical applications, only the fault recovery system needs to be certified, not the entire model, suggesting a practical path to trust.
Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation
A 2026 survey on world-model-based embodied AI traces threats across the model's lifecycle and warns that over-trusting world models can create 'predictive safety illusions', proposing evaluation protocols and defenses like robust grounding and uncertainty-aware prediction.