Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial
BackgroundGeneral awareness and exposure to generative artificial intelligence (AI) have increased recently. This transformative technology has the potential to create a more dynamic and engaging user experience in digital mental health interventions (DMHIs). However, if not...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-05-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e67365 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850150991591112704 |
|---|---|
| author | Timothy R Campellone Megan Flom Robert M Montgomery Lauren Bullard Maddison C Pirner Aaron Pavez Michelle Morales Devin Harper Catherine Oddy Tom O'Connor Jade Daniels Stephanie Eaneff Valerie L Forman-Hoffman Casey Sackett Alison Darcy |
| author_facet | Timothy R Campellone Megan Flom Robert M Montgomery Lauren Bullard Maddison C Pirner Aaron Pavez Michelle Morales Devin Harper Catherine Oddy Tom O'Connor Jade Daniels Stephanie Eaneff Valerie L Forman-Hoffman Casey Sackett Alison Darcy |
| author_sort | Timothy R Campellone |
| collection | DOAJ |
| description |
BackgroundGeneral awareness and exposure to generative artificial intelligence (AI) have increased recently. This transformative technology has the potential to create a more dynamic and engaging user experience in digital mental health interventions (DMHIs). However, if not appropriately used and controlled, it can introduce risks to users that may result in harm and erode trust. At the time of conducting this trial, there had not been a rigorous evaluation of an approach to safely implementing generative AI in a DMHI.
ObjectiveThis study aims to explore the user relationship, experience, safety, and technical guardrails of a DMHI using generative AI compared with a rules-based intervention.
MethodsWe conducted a 2-week exploratory randomized controlled trial (RCT) with 160 adult participants randomized to receive a generative AI (n=81) or rules-based (n=79) version of a conversation-based DMHI. Self-report measures of the user relationship (client satisfaction, working alliance bond, and accuracy of empathic listening and reflection) and experience (engagement metrics, adverse events, and technical guardrail success) were collected. Descriptions and validation of technical guardrails for handling user inputs (eg, detecting potentially concerning language and off-topic responses) and model outputs (eg, not providing medical advice and not providing a diagnosis) are provided, along with examples to illustrate how they worked. Safety monitoring was conducted throughout the trial for adverse events, and the success of technical guardrails created for the generative arm was assessed post trial.
ResultsIn general, the majority of measures of user relationship and experience appeared to be similar in both the generative and rules-based arms. The generative arm appeared to be more accurate at detecting and responding to user statements with empathy (98% accuracy vs 69%). There were no serious or device-related adverse events, and technical guardrails were shown to be 100% successful in posttrial review of generated statements. A majority of participants in both groups reported an increase in positive sentiment (62% and 66%) about AI at the end of the trial.
ConclusionsThis trial provides initial evidence that, with the right guardrails and process, generative AI can be successfully used in a digital mental health intervention (DMHI) while maintaining the user experience and relationship. It also provides an initial blueprint for approaches to technical and conversational guardrails that can be replicated to build a safe DMHI.
Trial RegistrationClinicalTrials.gov NCT05948670; https://clinicaltrials.gov/study/NCT05948670 |
| format | Article |
| id | doaj-art-5f5c9ad183474f47a0f96cfc92410366 |
| institution | OA Journals |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-5f5c9ad183474f47a0f96cfc924103662025-08-20T02:26:24ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-05-0127e6736510.2196/67365Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled TrialTimothy R Campellonehttps://orcid.org/0000-0002-8793-9576Megan Flomhttps://orcid.org/0000-0002-4896-1009Robert M Montgomeryhttps://orcid.org/0000-0001-9637-9753Lauren Bullardhttps://orcid.org/0000-0002-7368-2618Maddison C Pirnerhttps://orcid.org/0000-0002-3993-0606Aaron Pavezhttps://orcid.org/0009-0005-9358-7588Michelle Moraleshttps://orcid.org/0009-0005-0440-4373Devin Harperhttps://orcid.org/0009-0005-0955-8523Catherine Oddyhttps://orcid.org/0009-0000-9550-9560Tom O'Connorhttps://orcid.org/0009-0005-8214-0270Jade Danielshttps://orcid.org/0000-0001-8593-3712Stephanie Eaneffhttps://orcid.org/0000-0002-3657-5063Valerie L Forman-Hoffmanhttps://orcid.org/0000-0001-7885-8873Casey Sacketthttps://orcid.org/0009-0001-6314-0305Alison Darcyhttps://orcid.org/0000-0002-5082-7685 BackgroundGeneral awareness and exposure to generative artificial intelligence (AI) have increased recently. This transformative technology has the potential to create a more dynamic and engaging user experience in digital mental health interventions (DMHIs). However, if not appropriately used and controlled, it can introduce risks to users that may result in harm and erode trust. At the time of conducting this trial, there had not been a rigorous evaluation of an approach to safely implementing generative AI in a DMHI. ObjectiveThis study aims to explore the user relationship, experience, safety, and technical guardrails of a DMHI using generative AI compared with a rules-based intervention. MethodsWe conducted a 2-week exploratory randomized controlled trial (RCT) with 160 adult participants randomized to receive a generative AI (n=81) or rules-based (n=79) version of a conversation-based DMHI. Self-report measures of the user relationship (client satisfaction, working alliance bond, and accuracy of empathic listening and reflection) and experience (engagement metrics, adverse events, and technical guardrail success) were collected. Descriptions and validation of technical guardrails for handling user inputs (eg, detecting potentially concerning language and off-topic responses) and model outputs (eg, not providing medical advice and not providing a diagnosis) are provided, along with examples to illustrate how they worked. Safety monitoring was conducted throughout the trial for adverse events, and the success of technical guardrails created for the generative arm was assessed post trial. ResultsIn general, the majority of measures of user relationship and experience appeared to be similar in both the generative and rules-based arms. The generative arm appeared to be more accurate at detecting and responding to user statements with empathy (98% accuracy vs 69%). There were no serious or device-related adverse events, and technical guardrails were shown to be 100% successful in posttrial review of generated statements. A majority of participants in both groups reported an increase in positive sentiment (62% and 66%) about AI at the end of the trial. ConclusionsThis trial provides initial evidence that, with the right guardrails and process, generative AI can be successfully used in a digital mental health intervention (DMHI) while maintaining the user experience and relationship. It also provides an initial blueprint for approaches to technical and conversational guardrails that can be replicated to build a safe DMHI. Trial RegistrationClinicalTrials.gov NCT05948670; https://clinicaltrials.gov/study/NCT05948670https://www.jmir.org/2025/1/e67365 |
| spellingShingle | Timothy R Campellone Megan Flom Robert M Montgomery Lauren Bullard Maddison C Pirner Aaron Pavez Michelle Morales Devin Harper Catherine Oddy Tom O'Connor Jade Daniels Stephanie Eaneff Valerie L Forman-Hoffman Casey Sackett Alison Darcy Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial Journal of Medical Internet Research |
| title | Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial |
| title_full | Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial |
| title_fullStr | Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial |
| title_full_unstemmed | Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial |
| title_short | Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial |
| title_sort | safety and user experience of a generative artificial intelligence digital mental health intervention exploratory randomized controlled trial |
| url | https://www.jmir.org/2025/1/e67365 |
| work_keys_str_mv | AT timothyrcampellone safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT meganflom safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT robertmmontgomery safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT laurenbullard safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT maddisoncpirner safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT aaronpavez safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT michellemorales safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT devinharper safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT catherineoddy safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT tomoconnor safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT jadedaniels safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT stephanieeaneff safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT valerielformanhoffman safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT caseysackett safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial AT alisondarcy safetyanduserexperienceofagenerativeartificialintelligencedigitalmentalhealthinterventionexploratoryrandomizedcontrolledtrial |