Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation
BackgroundPrevious efforts to apply machine learning–based natural language processing to longitudinally collected social media data have shown promise in predicting suicide risk. ObjectiveOur primary objective was to externally validate our previous machine learn...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2024-12-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2024/1/e49927 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846140075697504256 |
|---|---|
| author | Zachary Kaminsky Robyn J McQuaid Kim GC Hellemans Zachary R Patterson Mysa Saad Robert L Gabrys Tetyana Kendzerska Alfonso Abizaid Rebecca Robillard |
| author_facet | Zachary Kaminsky Robyn J McQuaid Kim GC Hellemans Zachary R Patterson Mysa Saad Robert L Gabrys Tetyana Kendzerska Alfonso Abizaid Rebecca Robillard |
| author_sort | Zachary Kaminsky |
| collection | DOAJ |
| description |
BackgroundPrevious efforts to apply machine learning–based natural language processing to longitudinally collected social media data have shown promise in predicting suicide risk.
ObjectiveOur primary objective was to externally validate our previous machine learning algorithm, the Suicide Artificial Intelligence Prediction Heuristic (SAIPH), against external survey data in 2 independent cohorts. A second objective was to evaluate the efficacy of SAIPH as an indicator of changing suicidal ideation (SI) over time. The tertiary objective was to use SAIPH to evaluate factors important for improving or worsening suicidal trajectory on social media following suicidal mention.
MethodsTwitter (subsequently rebranded as X) timeline data from a student survey cohort and COVID-19 survey cohort were scored using SAIPH and compared to SI questions on the Beck Depression Inventory and the Self-Report version of the Quick Inventory of Depressive Symptomatology in 159 and 307 individuals, respectively. SAIPH was used to evaluate changing SI trajectory following suicidal mentions in 2 cohorts collected using the Twitter application programming interface.
ResultsAn interaction of the mean SAIPH score derived from 12 days of Twitter data before survey completion and the average number of posts per day was associated with quantitative SI metrics in each cohort (student survey cohort interaction β=.038, SD 0.014; F4,94=3.3, P=.01; and COVID-19 survey cohort interaction β=.0035, SD 0.0016; F4,493=2.9, P=.03). The slope of average daily SAIPH scores was associated with the change in SI scores within longitudinally followed individuals when evaluating periods of 2 weeks or less (ρ=0.27, P=.04). Using SAIPH as an indicator of changing SI, we evaluated SI trajectory in 2 cohorts with suicidal mentions, which identified that those with responses within 72 hours exhibit a significant negative association of the SAIPH score with time in the 3 weeks following suicidal mention (ρ=–0.52, P=.02).
ConclusionsTaken together, our results not only validate the association of SAIPH with perceived stress, SI, and changing SI over time but also generate novel methods to evaluate the effects of social media interactions on changing suicidal trajectory. |
| format | Article |
| id | doaj-art-93c485b45d0a406aa357998c23993d17 |
| institution | Kabale University |
| issn | 1438-8871 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-93c485b45d0a406aa357998c23993d172024-12-05T21:30:33ZengJMIR PublicationsJournal of Medical Internet Research1438-88712024-12-0126e4992710.2196/49927Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm ValidationZachary Kaminskyhttps://orcid.org/0000-0001-7917-1301Robyn J McQuaidhttps://orcid.org/0000-0003-4242-3135Kim GC Hellemanshttps://orcid.org/0000-0001-9169-7314Zachary R Pattersonhttps://orcid.org/0000-0001-7586-563XMysa Saadhttps://orcid.org/0000-0003-0611-559XRobert L Gabryshttps://orcid.org/0000-0002-9644-257XTetyana Kendzerskahttps://orcid.org/0000-0002-5301-1796Alfonso Abizaidhttps://orcid.org/0000-0001-9303-3511Rebecca Robillardhttps://orcid.org/0000-0002-1491-997X BackgroundPrevious efforts to apply machine learning–based natural language processing to longitudinally collected social media data have shown promise in predicting suicide risk. ObjectiveOur primary objective was to externally validate our previous machine learning algorithm, the Suicide Artificial Intelligence Prediction Heuristic (SAIPH), against external survey data in 2 independent cohorts. A second objective was to evaluate the efficacy of SAIPH as an indicator of changing suicidal ideation (SI) over time. The tertiary objective was to use SAIPH to evaluate factors important for improving or worsening suicidal trajectory on social media following suicidal mention. MethodsTwitter (subsequently rebranded as X) timeline data from a student survey cohort and COVID-19 survey cohort were scored using SAIPH and compared to SI questions on the Beck Depression Inventory and the Self-Report version of the Quick Inventory of Depressive Symptomatology in 159 and 307 individuals, respectively. SAIPH was used to evaluate changing SI trajectory following suicidal mentions in 2 cohorts collected using the Twitter application programming interface. ResultsAn interaction of the mean SAIPH score derived from 12 days of Twitter data before survey completion and the average number of posts per day was associated with quantitative SI metrics in each cohort (student survey cohort interaction β=.038, SD 0.014; F4,94=3.3, P=.01; and COVID-19 survey cohort interaction β=.0035, SD 0.0016; F4,493=2.9, P=.03). The slope of average daily SAIPH scores was associated with the change in SI scores within longitudinally followed individuals when evaluating periods of 2 weeks or less (ρ=0.27, P=.04). Using SAIPH as an indicator of changing SI, we evaluated SI trajectory in 2 cohorts with suicidal mentions, which identified that those with responses within 72 hours exhibit a significant negative association of the SAIPH score with time in the 3 weeks following suicidal mention (ρ=–0.52, P=.02). ConclusionsTaken together, our results not only validate the association of SAIPH with perceived stress, SI, and changing SI over time but also generate novel methods to evaluate the effects of social media interactions on changing suicidal trajectory.https://www.jmir.org/2024/1/e49927 |
| spellingShingle | Zachary Kaminsky Robyn J McQuaid Kim GC Hellemans Zachary R Patterson Mysa Saad Robert L Gabrys Tetyana Kendzerska Alfonso Abizaid Rebecca Robillard Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation Journal of Medical Internet Research |
| title | Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation |
| title_full | Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation |
| title_fullStr | Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation |
| title_full_unstemmed | Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation |
| title_short | Machine Learning–Based Suicide Risk Prediction Model for Suicidal Trajectory on Social Media Following Suicidal Mentions: Independent Algorithm Validation |
| title_sort | machine learning based suicide risk prediction model for suicidal trajectory on social media following suicidal mentions independent algorithm validation |
| url | https://www.jmir.org/2024/1/e49927 |
| work_keys_str_mv | AT zacharykaminsky machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT robynjmcquaid machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT kimgchellemans machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT zacharyrpatterson machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT mysasaad machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT robertlgabrys machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT tetyanakendzerska machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT alfonsoabizaid machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation AT rebeccarobillard machinelearningbasedsuicideriskpredictionmodelforsuicidaltrajectoryonsocialmediafollowingsuicidalmentionsindependentalgorithmvalidation |