Can AI companions actually engage users and expand access to mental health support?
Yes, the evidence shows that well-designed AI companions can attract and retain users at scale, which is a meaningful step toward addressing the global shortage of mental health professionals. In a real-world study of Headspace's purpose-built AI tool (Ebb), over 393,000 members used the tool, and the updated version (2.0) doubled the proportion of users who returned for a second session within a week—from 28.5% to 50.8% [2]. That means half of new users came back, which is strong for a digital tool. Users also rated conversations positively 93.5% of the time [2]. In a separate diagnostic study, an AI assistant called TalkToAlba conducted clinical interviews that actually outperformed standard rating scales in identifying self-reported clinician-diagnosed disorders, and most participants rated the AI as highly empathic and supportive [5]. These findings suggest that AI can feel helpful and engaging to users, and that it can be a scalable, low-cost way to reach people who might otherwise not seek care.
What are the main risks that make AI companions not ready for clinical use?
The core problem is that current AI systems lack the clinical judgment, crisis management, and accountability required for safe mental health care. A scoping review of 101 articles on conversational AI in mental health found that safety and harm was the most common ethical concern, discussed in over half (51.5%) of the papers, with top issues being suicidality, harmful suggestions, and risk of user dependency [1]. Privacy and confidentiality were flagged in 61.4% of articles, and effectiveness was questioned in 37.6% [1]. These are not minor issues—they are fundamental to whether a tool can be trusted in a crisis. A direct test of ChatGPT with three simulated patients showed that while it gave reasonable advice for a simple case, its recommendations became 'inappropriate, even dangerous' as the clinical complexity increased, because it could not ask follow-up questions or apply critical thinking [3]. Another analysis warned that ChatGPT lacks real-time fact-checking and can produce misleading information that may influence a person's thinking, potentially worsening mental health [4]. Together, these studies converge on a clear warning: AI companions can engage users, but they are not yet safe or reliable enough to replace or operate alongside human clinicians without rigorous safeguards.
What would need to change for AI companions to be ready for clinical or public health use?
The research points to three essential requirements: safety-by-design, transparent labeling, and human oversight. The Headspace study demonstrated that a purpose-built tool with clinically backed safety mechanisms—monitoring for overuse, detecting risk, and flagging needs for escalation—can achieve high engagement while maintaining a safety net [2]. The authors explicitly recommend transparent labeling of intended use, benefits, and limitations, plus child and adolescent safeguards [2]. The ethical review adds that accountability must be clearly assigned, and that AI should complement, not replace, human therapists [1]. The diagnostic study suggests that AI can be a valuable complement to traditional methods, but only when used as a screening or support tool, not as a standalone clinician [5]. In short, the path to readiness requires: (1) rigorous validation in real clinical settings, (2) built-in crisis detection and escalation protocols, (3) clear communication to users about what the AI can and cannot do, and (4) continued human oversight. None of the studies here suggest that current AI meets all these criteria.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2024 to 2026, 5 from 2024 or later, 2 in Q1 journals, collectively cited 185 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 49 papers retrieved from a database of over 500 million.
Sources used in this answer
Exploring the Ethical Challenges of Conversational AI in Mental Health Care: Scoping Review
A scoping review of 101 articles found that safety and harm (51.5% of articles) and privacy/confidentiality (61.4%) were the most common ethical concerns, with suicidality, harmful suggestions, and user dependency as top issues.
Real-World Use of a Mental Health AI Companion: Multiple Methods Study.
A real-world study of Headspace's AI companion (n=393,969 members) showed that the updated version achieved 50.8% 7-day retention and 93.5% positive conversation ratings, but users emphasized it should complement, not replace, human care.
ChatGPT is not ready yet for use in providing mental health assessment and interventions
In simulated patient scenarios, ChatGPT gave appropriate advice for a simple case but produced 'inappropriate, even dangerous' recommendations as clinical complexity increased, due to inability to ask follow-up questions or apply critical thinking.
ChatGPT and mental health: Friends or foes?
A review highlighted that ChatGPT lacks real-time fact-checking, can produce misleading information, and poses risks of bias and privacy violations, recommending education, stronger privacy measures, and updated ethical standards.
Generative AI-assisted clinical interviewing of mental health.
A study of an AI diagnostic assistant (TalkToAlba, n=303) found it achieved higher agreement, sensitivity, and specificity than standard rating scales in identifying self-reported clinician-diagnosed disorders, and most users rated it as highly empathic.
