Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study
BackgroundTimely and accurate prediction of short-term mortality is critical in intensive care units (ICUs), where patients’ conditions change rapidly. Traditional scoring systems, such as the Simplified Acute Physiology Score and Acute Physiology and Chronic Health Evaluatio...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-04-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e69293 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850155294710038528 |
|---|---|
| author | Zhuo Zheng Jiawei Luo Yingchao Zhu Lei Du Lan Lan Xiaobo Zhou Xiaoyan Yang Shixin Huang |
| author_facet | Zhuo Zheng Jiawei Luo Yingchao Zhu Lei Du Lan Lan Xiaobo Zhou Xiaoyan Yang Shixin Huang |
| author_sort | Zhuo Zheng |
| collection | DOAJ |
| description |
BackgroundTimely and accurate prediction of short-term mortality is critical in intensive care units (ICUs), where patients’ conditions change rapidly. Traditional scoring systems, such as the Simplified Acute Physiology Score and Acute Physiology and Chronic Health Evaluation, rely on static variables collected within the first 24 hours of admission and do not account for continuously evolving clinical states. These systems lack real-time adaptability, interpretability, and generalizability. With the increasing availability of high-frequency electronic medical record (EMR) data, machine learning (ML) approaches have emerged as powerful tools to model complex temporal patterns and support dynamic clinical decision-making. However, existing models are often limited by their inability to handle irregular sampling and missing values, and many lack rigorous external validation across institutions.
ObjectiveWe aimed to develop a real-time, interpretable risk prediction model that continuously assesses ICU patient mortality using irregular, longitudinal EMR data, with improved performance and generalizability over traditional static scoring systems.
MethodsA time-aware bidirectional attention-based long short-term memory (TBAL) model was developed using EMR data from the MIMIC-IV (Medical Information Mart for Intensive Care) and eICU Collaborative Research Database (eICU-CRD) databases, comprising 176,344 ICU stays. The model incorporated dynamic variables, including vital signs, laboratory results, and medication data, updated hourly, to perform static and continuous mortality risk assessments. External cross-validation and subgroup sensitivity analyses were conducted to evaluate robustness and fairness. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, and F1-score. Interpretability was enhanced using integrated gradients to identify key predictors.
ResultsFor the static 12-hour to 1-day mortality prediction task, the TBAL model achieved AUROCs of 95.9 (95% CI 94.2-97.5) and 93.3 (95% CI 91.5-95.3) and AUPRCs of 48.5 and 21.6 in MIMIC-IV and eICU-CRD, respectively. Accuracy and F1-scores reached 94.1 and 46.7 in MIMIC-IV and 92.2 and 28.1 in eICU-CRD. In dynamic prediction tasks, AUROCs reached 93.6 (95% CI 93.2-93.9) and 91.9 (95% CI 91.6-92.1), with AUPRCs of 41.3 and 50, respectively. The model maintained high recall for positive cases (82.6% and 79.1% in MIMIC-IV and eICU-CRD). Cross-database validation yielded AUROCs of 81.3 and 76.1, confirming generalizability. Subgroup analysis showed stable performance across age, sex, and severity strata, with top predictors including lactate, vasopressor use, and Glasgow Coma Scale score.
ConclusionsThe TBAL model offers a robust, interpretable, and generalizable solution for dynamic real-time mortality risk prediction in ICU patients. Its ability to adapt to irregular temporal patterns and to provide hourly updated predictions positions it as a promising decision-support tool. Future work should validate its utility in prospective clinical trials and investigate its integration into real-world ICU workflows to enhance patient outcomes. |
| format | Article |
| id | doaj-art-a7a8ca77d7aa400988d6b485a134d501 |
| institution | OA Journals |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-a7a8ca77d7aa400988d6b485a134d5012025-08-20T02:24:58ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-04-0127e6929310.2196/69293Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective StudyZhuo Zhenghttps://orcid.org/0009-0001-8660-466XJiawei Luohttps://orcid.org/0000-0003-1617-3224Yingchao Zhuhttps://orcid.org/0000-0003-2945-0163Lei Duhttps://orcid.org/0000-0001-9505-085XLan Lanhttps://orcid.org/0000-0003-2161-4936Xiaobo Zhouhttps://orcid.org/0000-0001-7191-6495Xiaoyan Yanghttps://orcid.org/0000-0003-3293-6201Shixin Huanghttps://orcid.org/0009-0001-7931-7056 BackgroundTimely and accurate prediction of short-term mortality is critical in intensive care units (ICUs), where patients’ conditions change rapidly. Traditional scoring systems, such as the Simplified Acute Physiology Score and Acute Physiology and Chronic Health Evaluation, rely on static variables collected within the first 24 hours of admission and do not account for continuously evolving clinical states. These systems lack real-time adaptability, interpretability, and generalizability. With the increasing availability of high-frequency electronic medical record (EMR) data, machine learning (ML) approaches have emerged as powerful tools to model complex temporal patterns and support dynamic clinical decision-making. However, existing models are often limited by their inability to handle irregular sampling and missing values, and many lack rigorous external validation across institutions. ObjectiveWe aimed to develop a real-time, interpretable risk prediction model that continuously assesses ICU patient mortality using irregular, longitudinal EMR data, with improved performance and generalizability over traditional static scoring systems. MethodsA time-aware bidirectional attention-based long short-term memory (TBAL) model was developed using EMR data from the MIMIC-IV (Medical Information Mart for Intensive Care) and eICU Collaborative Research Database (eICU-CRD) databases, comprising 176,344 ICU stays. The model incorporated dynamic variables, including vital signs, laboratory results, and medication data, updated hourly, to perform static and continuous mortality risk assessments. External cross-validation and subgroup sensitivity analyses were conducted to evaluate robustness and fairness. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, and F1-score. Interpretability was enhanced using integrated gradients to identify key predictors. ResultsFor the static 12-hour to 1-day mortality prediction task, the TBAL model achieved AUROCs of 95.9 (95% CI 94.2-97.5) and 93.3 (95% CI 91.5-95.3) and AUPRCs of 48.5 and 21.6 in MIMIC-IV and eICU-CRD, respectively. Accuracy and F1-scores reached 94.1 and 46.7 in MIMIC-IV and 92.2 and 28.1 in eICU-CRD. In dynamic prediction tasks, AUROCs reached 93.6 (95% CI 93.2-93.9) and 91.9 (95% CI 91.6-92.1), with AUPRCs of 41.3 and 50, respectively. The model maintained high recall for positive cases (82.6% and 79.1% in MIMIC-IV and eICU-CRD). Cross-database validation yielded AUROCs of 81.3 and 76.1, confirming generalizability. Subgroup analysis showed stable performance across age, sex, and severity strata, with top predictors including lactate, vasopressor use, and Glasgow Coma Scale score. ConclusionsThe TBAL model offers a robust, interpretable, and generalizable solution for dynamic real-time mortality risk prediction in ICU patients. Its ability to adapt to irregular temporal patterns and to provide hourly updated predictions positions it as a promising decision-support tool. Future work should validate its utility in prospective clinical trials and investigate its integration into real-world ICU workflows to enhance patient outcomes.https://www.jmir.org/2025/1/e69293 |
| spellingShingle | Zhuo Zheng Jiawei Luo Yingchao Zhu Lei Du Lan Lan Xiaobo Zhou Xiaoyan Yang Shixin Huang Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study Journal of Medical Internet Research |
| title | Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study |
| title_full | Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study |
| title_fullStr | Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study |
| title_full_unstemmed | Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study |
| title_short | Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study |
| title_sort | development and validation of a dynamic real time risk prediction model for intensive care units patients based on longitudinal irregular data multicenter retrospective study |
| url | https://www.jmir.org/2025/1/e69293 |
| work_keys_str_mv | AT zhuozheng developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT jiaweiluo developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT yingchaozhu developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT leidu developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT lanlan developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT xiaobozhou developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT xiaoyanyang developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy AT shixinhuang developmentandvalidationofadynamicrealtimeriskpredictionmodelforintensivecareunitspatientsbasedonlongitudinalirregulardatamulticenterretrospectivestudy |