Artificial intelligence chatbots as sources for patient education material on child abuse
Background: The World Health Organization defines childhood maltreatment as any form of abuse or neglect affecting children under 18 years of age that can cause actual or potential harm. Child abuse is a form of interpersonal trauma that can critically impact neurodevelopment and increase the risk o...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-07-01
|
| Series: | Child Protection and Practice |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2950193825000749 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850118645837987840 |
|---|---|
| author | Lily Nguyen Viet Tran Joy Li Denise Baughn Joseph Shotwell Kimberly Gushanas Sayyeda Hasan Lisa Falls Rocksheng Zhong |
| author_facet | Lily Nguyen Viet Tran Joy Li Denise Baughn Joseph Shotwell Kimberly Gushanas Sayyeda Hasan Lisa Falls Rocksheng Zhong |
| author_sort | Lily Nguyen |
| collection | DOAJ |
| description | Background: The World Health Organization defines childhood maltreatment as any form of abuse or neglect affecting children under 18 years of age that can cause actual or potential harm. Child abuse is a form of interpersonal trauma that can critically impact neurodevelopment and increase the risk of developing psychiatric disorders. With the increasing power and accessibility of artificial intelligence (AI) large language models, patients may turn to these platforms as sources of medical information. To date, no studies have evaluated the use of AI in creating patient education materials in childhood maltreatment and the field of psychiatry. Methods: Eight questions on child abuse from the National Child Traumatic Stress Network (NCTSN) were input into ChatGPT, Google Gemini, and Microsoft Copilot. A team of child psychiatrists and a pediatric psychologist reviewed and scored the responses by NCTSN and each AI, assessing quality, understandability, and actionability. Secondary outcomes included misinformation, readability, word count, and top references. Results: The analysis of 32 responses showed good quality (mean DISCERN score 51.7) and moderate understandability (mean PEMAT 76.5 %). However, actionability was poor (mean PEMAT 64 %). Responses averaged a tenth-grade reading level, with ChatGPT being more difficult to read than NCTSN. AI-generated responses were significantly longer (p < 0.001). Conclusions: Findings of this study suggest that AI chatbots may currently be able to provide accurate, quality information on child abuse comparable to authoritative sources, albeit of significantly greater length. However, all sources lack actionability and exceed recommended reading levels, which limits effectiveness. These constraints suggest that AI chatbots should supplement rather than replace primary medical information sources. Urgent efforts are needed to improve the accessibility, readability, and actionability of patient education materials generated by AI and standardized sources on topics like child abuse and neglect. |
| format | Article |
| id | doaj-art-2628cf7b9c4445808cf2d257b2d105df |
| institution | OA Journals |
| issn | 2950-1938 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Child Protection and Practice |
| spelling | doaj-art-2628cf7b9c4445808cf2d257b2d105df2025-08-20T02:35:48ZengElsevierChild Protection and Practice2950-19382025-07-01510016710.1016/j.chipro.2025.100167Artificial intelligence chatbots as sources for patient education material on child abuseLily Nguyen0Viet Tran1Joy Li2Denise Baughn3Joseph Shotwell4Kimberly Gushanas5Sayyeda Hasan6Lisa Falls7Rocksheng Zhong8John Sealy School of Medicine, University of Texas Medical Branch, Galveston, TX, USAJohn Sealy School of Medicine, University of Texas Medical Branch, Galveston, TX, USA; Corresponding author.John Sealy School of Medicine, University of Texas Medical Branch, Galveston, TX, USADepartment of Psychiatry and Behavioral Sciences, University of Texas Medical Branch, Galveston, TX, USADepartment of Psychiatry and Behavioral Sciences, University of Texas Medical Branch, Galveston, TX, USADepartment of Psychiatry and Behavioral Sciences, University of Texas Medical Branch, Galveston, TX, USADepartment of Psychiatry and Behavioral Sciences, University of Texas Medical Branch, Galveston, TX, USADepartment of Psychiatry and Behavioral Sciences, University of Texas Medical Branch, Galveston, TX, USADepartment of Psychiatry and Behavioral Sciences, University of Texas Medical Branch, Galveston, TX, USABackground: The World Health Organization defines childhood maltreatment as any form of abuse or neglect affecting children under 18 years of age that can cause actual or potential harm. Child abuse is a form of interpersonal trauma that can critically impact neurodevelopment and increase the risk of developing psychiatric disorders. With the increasing power and accessibility of artificial intelligence (AI) large language models, patients may turn to these platforms as sources of medical information. To date, no studies have evaluated the use of AI in creating patient education materials in childhood maltreatment and the field of psychiatry. Methods: Eight questions on child abuse from the National Child Traumatic Stress Network (NCTSN) were input into ChatGPT, Google Gemini, and Microsoft Copilot. A team of child psychiatrists and a pediatric psychologist reviewed and scored the responses by NCTSN and each AI, assessing quality, understandability, and actionability. Secondary outcomes included misinformation, readability, word count, and top references. Results: The analysis of 32 responses showed good quality (mean DISCERN score 51.7) and moderate understandability (mean PEMAT 76.5 %). However, actionability was poor (mean PEMAT 64 %). Responses averaged a tenth-grade reading level, with ChatGPT being more difficult to read than NCTSN. AI-generated responses were significantly longer (p < 0.001). Conclusions: Findings of this study suggest that AI chatbots may currently be able to provide accurate, quality information on child abuse comparable to authoritative sources, albeit of significantly greater length. However, all sources lack actionability and exceed recommended reading levels, which limits effectiveness. These constraints suggest that AI chatbots should supplement rather than replace primary medical information sources. Urgent efforts are needed to improve the accessibility, readability, and actionability of patient education materials generated by AI and standardized sources on topics like child abuse and neglect.http://www.sciencedirect.com/science/article/pii/S2950193825000749 |
| spellingShingle | Lily Nguyen Viet Tran Joy Li Denise Baughn Joseph Shotwell Kimberly Gushanas Sayyeda Hasan Lisa Falls Rocksheng Zhong Artificial intelligence chatbots as sources for patient education material on child abuse Child Protection and Practice |
| title | Artificial intelligence chatbots as sources for patient education material on child abuse |
| title_full | Artificial intelligence chatbots as sources for patient education material on child abuse |
| title_fullStr | Artificial intelligence chatbots as sources for patient education material on child abuse |
| title_full_unstemmed | Artificial intelligence chatbots as sources for patient education material on child abuse |
| title_short | Artificial intelligence chatbots as sources for patient education material on child abuse |
| title_sort | artificial intelligence chatbots as sources for patient education material on child abuse |
| url | http://www.sciencedirect.com/science/article/pii/S2950193825000749 |
| work_keys_str_mv | AT lilynguyen artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT viettran artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT joyli artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT denisebaughn artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT josephshotwell artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT kimberlygushanas artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT sayyedahasan artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT lisafalls artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse AT rockshengzhong artificialintelligencechatbotsassourcesforpatienteducationmaterialonchildabuse |