Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation
BackgroundDelirium is common in hospitalized patients and is correlated with increased morbidity and mortality. Despite this, delirium is underdiagnosed, and many institutions do not have sufficient resources to consistently apply effective screening and prevention....
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-04-01
|
| Series: | JMIR Medical Informatics |
| Online Access: | https://medinform.jmir.org/2025/1/e60442 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850200256001605632 |
|---|---|
| author | Kendrick Matthew Shaw Yu-Ping Shao Manohar Ghanta Valdery Moura Junior Eyal Y Kimchi Timothy T Houle Oluwaseun Akeju Michael Brandon Westover |
| author_facet | Kendrick Matthew Shaw Yu-Ping Shao Manohar Ghanta Valdery Moura Junior Eyal Y Kimchi Timothy T Houle Oluwaseun Akeju Michael Brandon Westover |
| author_sort | Kendrick Matthew Shaw |
| collection | DOAJ |
| description |
BackgroundDelirium is common in hospitalized patients and is correlated with increased morbidity and mortality. Despite this, delirium is underdiagnosed, and many institutions do not have sufficient resources to consistently apply effective screening and prevention.
ObjectiveThis study aims to develop a machine learning algorithm to identify patients at the highest risk of delirium in the hospital each day in an automated fashion based on data available in the electronic medical record, reducing the barrier to large-scale delirium screening.
MethodsWe developed and compared multiple machine learning models on a retrospective dataset of all hospitalized adult patients with recorded Confusion Assessment Method (CAM) screens at a major academic medical center from April 2, 2016, to January 16, 2019, comprising 23,006 patients. The patient’s age, gender, and all available laboratory values, vital signs, prior CAM screens, and medication administrations were used as potential predictors. Four machine learning approaches were investigated: logistic regression with L1-regularization, multilayer perceptrons, random forests, and boosted trees. Model development used 80% of the patients; the remaining 20% was reserved for testing the final models. Laboratory values, vital signs, medications, gender, and age were used to predict a positive CAM screen in the next 24 hours.
ResultsThe boosted tree model achieved the greatest predictive power, with an area under the receiver operator characteristic curve (AUROC) of 0.92 (95% CI 0.913-9.22), followed by the random forest (AUROC 0.91, 95% CI 0.909-0.918), multilayer perceptron (AUROC 0.86, 95% CI 0.850-0.861), and logistic regression (AUROC 0.85, 95% CI 0.841-0.852). These AUROCs decreased to 0.78-0.82 and 0.74-0.80 when limited to patients who currently do not or never have had delirium, respectively.
ConclusionsA boosted tree machine learning model was able to identify hospitalized patients at elevated risk for delirium in the next 24 hours. This may allow for automated delirium risk screening and more precise targeting of proven and investigational interventions to prevent delirium. |
| format | Article |
| id | doaj-art-adcfc5cb06614bb79cc497a9ae7f937d |
| institution | OA Journals |
| issn | 2291-9694 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Informatics |
| spelling | doaj-art-adcfc5cb06614bb79cc497a9ae7f937d2025-08-20T02:12:24ZengJMIR PublicationsJMIR Medical Informatics2291-96942025-04-0113e6044210.2196/60442Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and ValidationKendrick Matthew Shawhttps://orcid.org/0000-0001-6793-9600Yu-Ping Shaohttps://orcid.org/0000-0002-2265-1061Manohar Ghantahttps://orcid.org/0009-0004-8488-3644Valdery Moura Juniorhttps://orcid.org/0000-0001-5735-9143Eyal Y Kimchihttps://orcid.org/0000-0003-4327-1102Timothy T Houlehttps://orcid.org/0000-0001-9584-5580Oluwaseun Akejuhttps://orcid.org/0000-0002-6740-1250Michael Brandon Westoverhttps://orcid.org/0000-0003-4803-312X BackgroundDelirium is common in hospitalized patients and is correlated with increased morbidity and mortality. Despite this, delirium is underdiagnosed, and many institutions do not have sufficient resources to consistently apply effective screening and prevention. ObjectiveThis study aims to develop a machine learning algorithm to identify patients at the highest risk of delirium in the hospital each day in an automated fashion based on data available in the electronic medical record, reducing the barrier to large-scale delirium screening. MethodsWe developed and compared multiple machine learning models on a retrospective dataset of all hospitalized adult patients with recorded Confusion Assessment Method (CAM) screens at a major academic medical center from April 2, 2016, to January 16, 2019, comprising 23,006 patients. The patient’s age, gender, and all available laboratory values, vital signs, prior CAM screens, and medication administrations were used as potential predictors. Four machine learning approaches were investigated: logistic regression with L1-regularization, multilayer perceptrons, random forests, and boosted trees. Model development used 80% of the patients; the remaining 20% was reserved for testing the final models. Laboratory values, vital signs, medications, gender, and age were used to predict a positive CAM screen in the next 24 hours. ResultsThe boosted tree model achieved the greatest predictive power, with an area under the receiver operator characteristic curve (AUROC) of 0.92 (95% CI 0.913-9.22), followed by the random forest (AUROC 0.91, 95% CI 0.909-0.918), multilayer perceptron (AUROC 0.86, 95% CI 0.850-0.861), and logistic regression (AUROC 0.85, 95% CI 0.841-0.852). These AUROCs decreased to 0.78-0.82 and 0.74-0.80 when limited to patients who currently do not or never have had delirium, respectively. ConclusionsA boosted tree machine learning model was able to identify hospitalized patients at elevated risk for delirium in the next 24 hours. This may allow for automated delirium risk screening and more precise targeting of proven and investigational interventions to prevent delirium.https://medinform.jmir.org/2025/1/e60442 |
| spellingShingle | Kendrick Matthew Shaw Yu-Ping Shao Manohar Ghanta Valdery Moura Junior Eyal Y Kimchi Timothy T Houle Oluwaseun Akeju Michael Brandon Westover Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation JMIR Medical Informatics |
| title | Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation |
| title_full | Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation |
| title_fullStr | Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation |
| title_full_unstemmed | Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation |
| title_short | Daily Automated Prediction of Delirium Risk in Hospitalized Patients: Model Development and Validation |
| title_sort | daily automated prediction of delirium risk in hospitalized patients model development and validation |
| url | https://medinform.jmir.org/2025/1/e60442 |
| work_keys_str_mv | AT kendrickmatthewshaw dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT yupingshao dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT manoharghanta dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT valderymourajunior dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT eyalykimchi dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT timothythoule dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT oluwaseunakeju dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation AT michaelbrandonwestover dailyautomatedpredictionofdeliriumriskinhospitalizedpatientsmodeldevelopmentandvalidation |