Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study
BackgroundStroke is a globally prevalent disease that imposes a significant burden on health care systems and national economies. Accurate and rapid stroke diagnosis can substantially increase reperfusion rates, mitigate disability, and reduce mortality. However, there are co...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-02-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e67010 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850079299136126976 |
|---|---|
| author | Xiaowei Song Jiayi Wang Feifei He Wei Yin Weizhi Ma Jian Wu |
| author_facet | Xiaowei Song Jiayi Wang Feifei He Wei Yin Weizhi Ma Jian Wu |
| author_sort | Xiaowei Song |
| collection | DOAJ |
| description |
BackgroundStroke is a globally prevalent disease that imposes a significant burden on health care systems and national economies. Accurate and rapid stroke diagnosis can substantially increase reperfusion rates, mitigate disability, and reduce mortality. However, there are considerable discrepancies in the diagnosis and treatment of acute stroke.
ObjectiveThe aim of this study is to develop and validate a stroke diagnosis and prediction tool using ChatGLM-6B, which uses free-text information from electronic health records in conjunction with noncontrast computed tomography (NCCT) reports to enhance stroke detection and treatment.
MethodsA large language model (LLM) using ChatGLM-6B was proposed to facilitate stroke diagnosis by identifying optimal input combinations, using external tools, and applying instruction tuning and low-rank adaptation (LoRA) techniques. A dataset containing details of 1885 patients with and those without stroke from 2016 to 2024 was used for training and internal validation; another 335 patients from two hospitals were used as an external test set, including 230 patients from the training hospital but admitted at different periods, and 105 patients from another hospital.
ResultsThe LLM, which is based on clinical notes and NCCT, demonstrates exceptionally high accuracy in stroke diagnosis, achieving 99% in the internal validation dataset and 95.5% and 79.1% in two external test cohorts. It effectively distinguishes between ischemia and hemorrhage, with an accuracy of 100% in the validation dataset and 99.1% and 97.1% in the other test cohorts. In addition, it identifies large vessel occlusions (LVO) with an accuracy of 80% in the validation dataset and 88.6% and 83.3% in the other test cohorts. Furthermore, it screens patients eligible for intravenous thrombolysis (IVT) with an accuracy of 89.4% in the validation dataset and 60% and 80% in the other test cohorts.
ConclusionsWe developed an LLM that leverages clinical text and NCCT to identify strokes and guide recanalization therapy. While our results necessitate validation through widespread deployment, they hold the potential to enhance stroke identification and reduce reperfusion time. |
| format | Article |
| id | doaj-art-b29f0d03acb84fa3a132b171de38fdab |
| institution | DOAJ |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-b29f0d03acb84fa3a132b171de38fdab2025-08-20T02:45:15ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-02-0127e6701010.2196/67010Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation StudyXiaowei Songhttps://orcid.org/0000-0001-7707-7995Jiayi Wanghttps://orcid.org/0009-0009-3243-6873Feifei Hehttps://orcid.org/0009-0004-0218-6774Wei Yinhttps://orcid.org/0009-0000-1866-7071Weizhi Mahttps://orcid.org/0000-0001-5604-7527Jian Wuhttps://orcid.org/0000-0002-0943-314X BackgroundStroke is a globally prevalent disease that imposes a significant burden on health care systems and national economies. Accurate and rapid stroke diagnosis can substantially increase reperfusion rates, mitigate disability, and reduce mortality. However, there are considerable discrepancies in the diagnosis and treatment of acute stroke. ObjectiveThe aim of this study is to develop and validate a stroke diagnosis and prediction tool using ChatGLM-6B, which uses free-text information from electronic health records in conjunction with noncontrast computed tomography (NCCT) reports to enhance stroke detection and treatment. MethodsA large language model (LLM) using ChatGLM-6B was proposed to facilitate stroke diagnosis by identifying optimal input combinations, using external tools, and applying instruction tuning and low-rank adaptation (LoRA) techniques. A dataset containing details of 1885 patients with and those without stroke from 2016 to 2024 was used for training and internal validation; another 335 patients from two hospitals were used as an external test set, including 230 patients from the training hospital but admitted at different periods, and 105 patients from another hospital. ResultsThe LLM, which is based on clinical notes and NCCT, demonstrates exceptionally high accuracy in stroke diagnosis, achieving 99% in the internal validation dataset and 95.5% and 79.1% in two external test cohorts. It effectively distinguishes between ischemia and hemorrhage, with an accuracy of 100% in the validation dataset and 99.1% and 97.1% in the other test cohorts. In addition, it identifies large vessel occlusions (LVO) with an accuracy of 80% in the validation dataset and 88.6% and 83.3% in the other test cohorts. Furthermore, it screens patients eligible for intravenous thrombolysis (IVT) with an accuracy of 89.4% in the validation dataset and 60% and 80% in the other test cohorts. ConclusionsWe developed an LLM that leverages clinical text and NCCT to identify strokes and guide recanalization therapy. While our results necessitate validation through widespread deployment, they hold the potential to enhance stroke identification and reduce reperfusion time.https://www.jmir.org/2025/1/e67010 |
| spellingShingle | Xiaowei Song Jiayi Wang Feifei He Wei Yin Weizhi Ma Jian Wu Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study Journal of Medical Internet Research |
| title | Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study |
| title_full | Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study |
| title_fullStr | Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study |
| title_full_unstemmed | Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study |
| title_short | Stroke Diagnosis and Prediction Tool Using ChatGLM: Development and Validation Study |
| title_sort | stroke diagnosis and prediction tool using chatglm development and validation study |
| url | https://www.jmir.org/2025/1/e67010 |
| work_keys_str_mv | AT xiaoweisong strokediagnosisandpredictiontoolusingchatglmdevelopmentandvalidationstudy AT jiayiwang strokediagnosisandpredictiontoolusingchatglmdevelopmentandvalidationstudy AT feifeihe strokediagnosisandpredictiontoolusingchatglmdevelopmentandvalidationstudy AT weiyin strokediagnosisandpredictiontoolusingchatglmdevelopmentandvalidationstudy AT weizhima strokediagnosisandpredictiontoolusingchatglmdevelopmentandvalidationstudy AT jianwu strokediagnosisandpredictiontoolusingchatglmdevelopmentandvalidationstudy |