The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study
Abstract Background Improving media adherence to World Health Organization (WHO) guidelines is crucial for preventing suicidal behaviors in the general population. However, there is currently no valid, rapid, and effective method to evaluate the adherence to these guidelines. Methods This compar...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2025-01-01
|
| Series: | European Psychiatry |
| Subjects: | |
| Online Access: | https://www.cambridge.org/core/product/identifier/S0924933825100370/type/journal_article |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850166345071591424 |
|---|---|
| author | Zohar Elyospeh Bénédicte Nobile Inbar Levkovich Raphael Chancel Philippe Courtet Yossi Levi-Belz |
| author_facet | Zohar Elyospeh Bénédicte Nobile Inbar Levkovich Raphael Chancel Philippe Courtet Yossi Levi-Belz |
| author_sort | Zohar Elyospeh |
| collection | DOAJ |
| description |
Abstract
Background
Improving media adherence to World Health Organization (WHO) guidelines is crucial for preventing suicidal behaviors in the general population. However, there is currently no valid, rapid, and effective method to evaluate the adherence to these guidelines.
Methods
This comparative effectiveness study (January–August 2024) evaluated the ability of two artificial intelligence (AI) models (Claude Opus 3 and GPT-4O) to assess the adherence of media reports to WHO suicide-reporting guidelines. A total of 120 suicide-related articles (40 in English, 40 in Hebrew, and 40 in French) published within the past 5 years were sourced from prominent newspapers. Six trained human raters (two per language) independently evaluated articles based on a WHO guideline-based questionnaire addressing aspects, such as prominence, sensationalism, and prevention. The same articles were also processed using AI models. Intraclass correlation coefficients (ICCs) and Spearman correlations were calculated to assess agreement between human raters and AI models.
Results
Overall adherence to WHO guidelines was ~50% across all languages. Both AI models demonstrated strong agreement with human raters, with GPT-4O showing the highest agreement (ICC = 0.793 [0.702; 0.855]). The combined evaluations of GPT-4O and Claude Opus 3 yielded the highest reliability (ICC = 0.812 [0.731; 0.869]).
Conclusions
AI models can replicate human judgment in evaluating media adherence to WHO guidelines. However, they have limitations and should be used alongside human oversight. These findings may suggest that AI tools have the potential to enhance and promote responsible reporting practices among journalists and, thus, may support suicide prevention efforts globally.
|
| format | Article |
| id | doaj-art-ebef37b46f0d43f1bab44050afeae279 |
| institution | OA Journals |
| issn | 0924-9338 1778-3585 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | European Psychiatry |
| spelling | doaj-art-ebef37b46f0d43f1bab44050afeae2792025-08-20T02:21:29ZengCambridge University PressEuropean Psychiatry0924-93381778-35852025-01-016810.1192/j.eurpsy.2025.10037The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language studyZohar Elyospeh0Bénédicte Nobile1https://orcid.org/0000-0002-2570-6996Inbar Levkovich2https://orcid.org/0000-0003-1582-3889Raphael Chancel3https://orcid.org/0009-0009-3741-7508Philippe Courtet4https://orcid.org/0000-0002-6519-8586Yossi Levi-Belz5https://ror.org/02f009v59University of Haifa, Mount Carmel, Haifa, IsraelDepartment of Emergency Psychiatry and Acute Care, CHU Montpellier, Montpellier, France IGF, Univ. Montpellier, CNRS, INSERM, Montpellier, FranceFaculty of Education, https://ror.org/009st3569Tel Hai College, Upper Galilee, IsraelDepartment of Emergency Psychiatry and Acute Care, CHU Montpellier, Montpellier, FranceDepartment of Emergency Psychiatry and Acute Care, CHU Montpellier, Montpellier, France IGF, Univ. Montpellier, CNRS, INSERM, Montpellier, FranceLior Tsfaty Center for Suicide and Mental Pain Studies, https://ror.org/0361c8163Ruppin Academic Center, Emek Hefer, Israel Abstract Background Improving media adherence to World Health Organization (WHO) guidelines is crucial for preventing suicidal behaviors in the general population. However, there is currently no valid, rapid, and effective method to evaluate the adherence to these guidelines. Methods This comparative effectiveness study (January–August 2024) evaluated the ability of two artificial intelligence (AI) models (Claude Opus 3 and GPT-4O) to assess the adherence of media reports to WHO suicide-reporting guidelines. A total of 120 suicide-related articles (40 in English, 40 in Hebrew, and 40 in French) published within the past 5 years were sourced from prominent newspapers. Six trained human raters (two per language) independently evaluated articles based on a WHO guideline-based questionnaire addressing aspects, such as prominence, sensationalism, and prevention. The same articles were also processed using AI models. Intraclass correlation coefficients (ICCs) and Spearman correlations were calculated to assess agreement between human raters and AI models. Results Overall adherence to WHO guidelines was ~50% across all languages. Both AI models demonstrated strong agreement with human raters, with GPT-4O showing the highest agreement (ICC = 0.793 [0.702; 0.855]). The combined evaluations of GPT-4O and Claude Opus 3 yielded the highest reliability (ICC = 0.812 [0.731; 0.869]). Conclusions AI models can replicate human judgment in evaluating media adherence to WHO guidelines. However, they have limitations and should be used alongside human oversight. These findings may suggest that AI tools have the potential to enhance and promote responsible reporting practices among journalists and, thus, may support suicide prevention efforts globally. https://www.cambridge.org/core/product/identifier/S0924933825100370/type/journal_articleartificial intelligencemedianatural language processingsuicidesuicide prevention |
| spellingShingle | Zohar Elyospeh Bénédicte Nobile Inbar Levkovich Raphael Chancel Philippe Courtet Yossi Levi-Belz The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study European Psychiatry artificial intelligence media natural language processing suicide suicide prevention |
| title | The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study |
| title_full | The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study |
| title_fullStr | The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study |
| title_full_unstemmed | The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study |
| title_short | The role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide: A multisite, three-language study |
| title_sort | role of generative artificial intelligence in evaluating adherence to responsible press media reports on suicide a multisite three language study |
| topic | artificial intelligence media natural language processing suicide suicide prevention |
| url | https://www.cambridge.org/core/product/identifier/S0924933825100370/type/journal_article |
| work_keys_str_mv | AT zoharelyospeh theroleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT benedictenobile theroleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT inbarlevkovich theroleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT raphaelchancel theroleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT philippecourtet theroleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT yossilevibelz theroleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT zoharelyospeh roleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT benedictenobile roleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT inbarlevkovich roleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT raphaelchancel roleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT philippecourtet roleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy AT yossilevibelz roleofgenerativeartificialintelligenceinevaluatingadherencetoresponsiblepressmediareportsonsuicideamultisitethreelanguagestudy |