Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context
Summary: Background: Sepsis is a significant health burden on a global scale. Timely identification and treatment of sepsis can greatly improve patient outcomes, including survival rates. However, time-consuming laboratory results are often needed for screening sepsis. We aimed to develop a quick s...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-02-01
|
| Series: | EClinicalMedicine |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2589537024006278 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841545537492353024 |
|---|---|
| author | Shan Jiang Shuai Dai Yulin Li Xianlong Zhou Cheng Jiang Cong Tian Yana Yuan Chengwei Li Yan Zhao |
| author_facet | Shan Jiang Shuai Dai Yulin Li Xianlong Zhou Cheng Jiang Cong Tian Yana Yuan Chengwei Li Yan Zhao |
| author_sort | Shan Jiang |
| collection | DOAJ |
| description | Summary: Background: Sepsis is a significant health burden on a global scale. Timely identification and treatment of sepsis can greatly improve patient outcomes, including survival rates. However, time-consuming laboratory results are often needed for screening sepsis. We aimed to develop a quick sepsis screening tool (qSepsis) based on patients’ non-laboratory clinical data at the emergency department (ED) using machine learning (ML), and compare its performance with established clinical scores. Methods: This retrospective study included patients admitted to the ED of Zhongnan Hospital of Wuhan University (Wuhan, China) from 1/1/2015 to 5/31/2022. Patients who were under 18 years of age, had cardiopulmonary arrest upon arrival at the ED, or had missing and abnormal medical record data were excluded. The qSepsis was derived by three ML algorithms, including logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). To benchmark the existing clinical tools for assessing the risk of sepsis in the ED, qSepsis was compared with the Systemic Inflammatory Response Syndrome (SIRS), the Quick Sepsis-Related Organ Failure Assessment (qSOFA), and the Modified Early Warning Score (MEWS). The external validation was performed with the Medical Information Mart for Intensive Care IV ED database (United States), and adopted the same inclusion and exclusion criteria. The predictive power of qSepsis and other clinical scores was measured using the area under the receiver operating characteristic curve (AUROC). The primary outcome of the study was the diagnosis of sepsis in the ED based on the Sepsis 3.0 criteria, which served as the basis for developing the qSepsis tool. Findings: A total of 414,864 patients were finally included in the cohort (median ([IQR]) patient age, 43 (29, 60) years; 202,730 (48.87%) females, 212,134 (51.13%) males), and 200,089 in the external testing cohort (median (SD) patient age, 57 (39, 71) years; 107,427 (53.69%) females, 92,663 (46.31%) males). For internal testing, LR outperformed RF and XGB with an AUROC of 0.862 (95% CI, 0.855–0.869). In external testing, the AUROC decreased to 0.766 (95% CI, 0.758–0.774) for LR, 0.725 (95% CI, 0.717–0.733) for RF, and 0.735 (95% CI, 0.728–0.742) for XGB. In addition, the AUROC for the qSOFA, MEWS, and SIRS scores in external validation cohort were 0.579 (95% CI, 0.563–0.596), 0.600 (95% CI, 0.578–0.622), and 0.704 (95% CI, 0.683–0.725), respectively. The area under the precision–recall curve (AUPRC) for the qSepsis model was 0.213 (95% CI: 0.204–0.222). The AUPRC values for the other scores were as follows: SIRS, 0.071 (95% CI: 0.013–0.099); qSOFA, 0.096 (95% CI: 0.003–0.186); and MEWS, 0.083 (95% CI: 0.063–0.111). Interpretation: This retrospective study demonstrated that qSepsis had better predictive performance in terms of AUROC and area under the precision–recall curve (AUPRC) compared to existing assessment scores. It has the potential to be used in pre-hospital settings with limited access to laboratory tests and in the ED for quick screening of patients with sepsis. However, due to its low positive predictive value (PPV), the false alarms may increase in actual clinical practice. Funding: Transformation of Scientific and Technological Achievements Fund Project of Zhongnan Hospital of Wuhan University. |
| format | Article |
| id | doaj-art-c8373babf7b74c22abd8f8bf17426da1 |
| institution | Kabale University |
| issn | 2589-5370 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Elsevier |
| record_format | Article |
| series | EClinicalMedicine |
| spelling | doaj-art-c8373babf7b74c22abd8f8bf17426da12025-01-12T05:25:35ZengElsevierEClinicalMedicine2589-53702025-02-0180103048Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in contextShan Jiang0Shuai Dai1Yulin Li2Xianlong Zhou3Cheng Jiang4Cong Tian5Yana Yuan6Chengwei Li7Yan Zhao8Emergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Hubei Clinical Research Centre of Emergency and Resuscitation, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, ChinaEmergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Hubei Clinical Research Centre of Emergency and Resuscitation, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, ChinaEmergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Hubei Clinical Research Centre of Emergency and Resuscitation, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, ChinaEmergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Hubei Clinical Research Centre of Emergency and Resuscitation, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, ChinaEmergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Hubei Clinical Research Centre of Emergency and Resuscitation, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, ChinaPhilips Research China, Shanghai, ChinaPhilips Research China, Shanghai, ChinaInformation Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Corresponding author. Information Centre, Zhongnan Hospital of Wuhan University, Wuhan, 430071, China.Emergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Hubei Clinical Research Centre of Emergency and Resuscitation, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China; Corresponding author. Emergency Centre, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China.Summary: Background: Sepsis is a significant health burden on a global scale. Timely identification and treatment of sepsis can greatly improve patient outcomes, including survival rates. However, time-consuming laboratory results are often needed for screening sepsis. We aimed to develop a quick sepsis screening tool (qSepsis) based on patients’ non-laboratory clinical data at the emergency department (ED) using machine learning (ML), and compare its performance with established clinical scores. Methods: This retrospective study included patients admitted to the ED of Zhongnan Hospital of Wuhan University (Wuhan, China) from 1/1/2015 to 5/31/2022. Patients who were under 18 years of age, had cardiopulmonary arrest upon arrival at the ED, or had missing and abnormal medical record data were excluded. The qSepsis was derived by three ML algorithms, including logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). To benchmark the existing clinical tools for assessing the risk of sepsis in the ED, qSepsis was compared with the Systemic Inflammatory Response Syndrome (SIRS), the Quick Sepsis-Related Organ Failure Assessment (qSOFA), and the Modified Early Warning Score (MEWS). The external validation was performed with the Medical Information Mart for Intensive Care IV ED database (United States), and adopted the same inclusion and exclusion criteria. The predictive power of qSepsis and other clinical scores was measured using the area under the receiver operating characteristic curve (AUROC). The primary outcome of the study was the diagnosis of sepsis in the ED based on the Sepsis 3.0 criteria, which served as the basis for developing the qSepsis tool. Findings: A total of 414,864 patients were finally included in the cohort (median ([IQR]) patient age, 43 (29, 60) years; 202,730 (48.87%) females, 212,134 (51.13%) males), and 200,089 in the external testing cohort (median (SD) patient age, 57 (39, 71) years; 107,427 (53.69%) females, 92,663 (46.31%) males). For internal testing, LR outperformed RF and XGB with an AUROC of 0.862 (95% CI, 0.855–0.869). In external testing, the AUROC decreased to 0.766 (95% CI, 0.758–0.774) for LR, 0.725 (95% CI, 0.717–0.733) for RF, and 0.735 (95% CI, 0.728–0.742) for XGB. In addition, the AUROC for the qSOFA, MEWS, and SIRS scores in external validation cohort were 0.579 (95% CI, 0.563–0.596), 0.600 (95% CI, 0.578–0.622), and 0.704 (95% CI, 0.683–0.725), respectively. The area under the precision–recall curve (AUPRC) for the qSepsis model was 0.213 (95% CI: 0.204–0.222). The AUPRC values for the other scores were as follows: SIRS, 0.071 (95% CI: 0.013–0.099); qSOFA, 0.096 (95% CI: 0.003–0.186); and MEWS, 0.083 (95% CI: 0.063–0.111). Interpretation: This retrospective study demonstrated that qSepsis had better predictive performance in terms of AUROC and area under the precision–recall curve (AUPRC) compared to existing assessment scores. It has the potential to be used in pre-hospital settings with limited access to laboratory tests and in the ED for quick screening of patients with sepsis. However, due to its low positive predictive value (PPV), the false alarms may increase in actual clinical practice. Funding: Transformation of Scientific and Technological Achievements Fund Project of Zhongnan Hospital of Wuhan University.http://www.sciencedirect.com/science/article/pii/S2589537024006278SepsisMachine learningEmergency serviceElectronic health recordsClinical decision support systems |
| spellingShingle | Shan Jiang Shuai Dai Yulin Li Xianlong Zhou Cheng Jiang Cong Tian Yana Yuan Chengwei Li Yan Zhao Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context EClinicalMedicine Sepsis Machine learning Emergency service Electronic health records Clinical decision support systems |
| title | Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context |
| title_full | Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context |
| title_fullStr | Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context |
| title_full_unstemmed | Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context |
| title_short | Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning studyResearch in context |
| title_sort | development and validation of a screening tool for sepsis without laboratory results in the emergency department a machine learning studyresearch in context |
| topic | Sepsis Machine learning Emergency service Electronic health records Clinical decision support systems |
| url | http://www.sciencedirect.com/science/article/pii/S2589537024006278 |
| work_keys_str_mv | AT shanjiang developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT shuaidai developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT yulinli developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT xianlongzhou developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT chengjiang developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT congtian developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT yanayuan developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT chengweili developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext AT yanzhao developmentandvalidationofascreeningtoolforsepsiswithoutlaboratoryresultsintheemergencydepartmentamachinelearningstudyresearchincontext |