Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation
BackgroundFor patients with drug-resistant focal epilepsy, surgical resection of the epileptogenic zone (EZ) is an effective treatment to control seizures. Accurate localization of the EZ is crucial and is typically achieved through comprehensive presurgical approaches such a...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-05-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e69173 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850274565746327552 |
|---|---|
| author | Yaxi Luo Meng Jiao Neel Fotedar Jun-En Ding Ioannis Karakis Vikram R Rao Melissa Asmar Xiaochen Xian Orwa Aboud Yuxin Wen Jack J Lin Fang-Ming Hung Hai Sun Felix Rosenow Feng Liu |
| author_facet | Yaxi Luo Meng Jiao Neel Fotedar Jun-En Ding Ioannis Karakis Vikram R Rao Melissa Asmar Xiaochen Xian Orwa Aboud Yuxin Wen Jack J Lin Fang-Ming Hung Hai Sun Felix Rosenow Feng Liu |
| author_sort | Yaxi Luo |
| collection | DOAJ |
| description |
BackgroundFor patients with drug-resistant focal epilepsy, surgical resection of the epileptogenic zone (EZ) is an effective treatment to control seizures. Accurate localization of the EZ is crucial and is typically achieved through comprehensive presurgical approaches such as seizure semiology interpretation, electroencephalography (EEG), magnetic resonance imaging (MRI), and intracranial EEG (iEEG). However, interpreting seizure semiology is challenging because it heavily relies on expert knowledge. The semiologies are often inconsistent and incoherent, leading to variability and potential limitations in presurgical evaluation. To overcome these challenges, advanced technologies like large language models (LLMs)—with ChatGPT being a notable example—offer valuable tools for analyzing complex textual information, making them well-suited to interpret detailed seizure semiology descriptions and accurately localize the EZ.
ObjectiveThis study evaluates the clinical value of ChatGPT for interpreting seizure semiology to localize EZs in presurgical assessments for patients with focal epilepsy and compares its performance with that of epileptologists.
MethodsWe compiled 2 data cohorts: a publicly sourced cohort of 852 semiology-EZ pairs from 193 peer-reviewed journal publications and a private cohort of 184 semiology-EZ pairs collected from Far Eastern Memorial Hospital (FEMH) in Taiwan. ChatGPT was evaluated to predict the most likely EZ locations using 2 prompt methods: zero-shot prompting (ZSP) and few-shot prompting (FSP). To compare the performance of ChatGPT, 8 epileptologists were recruited to participate in an online survey to interpret 100 randomly selected semiology records. The responses from ChatGPT and epileptologists were compared using 3 metrics: regional sensitivity (RSens), weighted sensitivity (WSens), and net positive inference rate (NPIR).
ResultsIn the publicly sourced cohort, ChatGPT demonstrated high RSens reliability, achieving 80% to 90% for the frontal and temporal lobes; 20% to 40% for the parietal lobe, occipital lobe, and insular cortex; and only 3% for the cingulate cortex. The WSens, which accounts for biased data distribution, consistently exceeded 67%, while the mean NPIR remained around 0. These evaluation results based on the private FEMH cohort are consistent with those from the publicly sourced cohort. A group t test with 1000 bootstrap samples revealed that ChatGPT-4 significantly outperformed epileptologists in RSens for the most frequently implicated EZs, such as the frontal and temporal lobes (P<.001). Additionally, ChatGPT-4 demonstrated superior overall performance in WSens (P<.001). However, no significant differences were observed between ChatGPT and the epileptologists in NPIR, highlighting comparable performance in this metric.
ConclusionsChatGPT demonstrated clinical value as a tool to assist decision-making during epilepsy preoperative workups. With ongoing advancements in LLMs, their reliability and accuracy are anticipated to improve. |
| format | Article |
| id | doaj-art-2255e90a4d71411b82f2015152f28c5b |
| institution | OA Journals |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-2255e90a4d71411b82f2015152f28c5b2025-08-20T01:51:06ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-05-0127e6917310.2196/69173Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology InterpretationYaxi Luohttps://orcid.org/0009-0009-1391-0146Meng Jiaohttps://orcid.org/0009-0009-1585-0511Neel Fotedarhttps://orcid.org/0000-0003-0424-7767Jun-En Dinghttps://orcid.org/0000-0002-1233-138XIoannis Karakishttps://orcid.org/0000-0001-5122-7211Vikram R Raohttps://orcid.org/0000-0002-6389-2638Melissa Asmarhttps://orcid.org/0009-0008-0220-0264Xiaochen Xianhttps://orcid.org/0000-0001-7099-2488Orwa Aboudhttps://orcid.org/0000-0002-7916-1629Yuxin Wenhttps://orcid.org/0000-0002-2352-5622Jack J Linhttps://orcid.org/0000-0003-1304-227XFang-Ming Hunghttps://orcid.org/0000-0003-3501-5459Hai Sunhttps://orcid.org/0000-0001-8878-8776Felix Rosenowhttps://orcid.org/0000-0002-3989-7471Feng Liuhttps://orcid.org/0000-0002-5225-8199 BackgroundFor patients with drug-resistant focal epilepsy, surgical resection of the epileptogenic zone (EZ) is an effective treatment to control seizures. Accurate localization of the EZ is crucial and is typically achieved through comprehensive presurgical approaches such as seizure semiology interpretation, electroencephalography (EEG), magnetic resonance imaging (MRI), and intracranial EEG (iEEG). However, interpreting seizure semiology is challenging because it heavily relies on expert knowledge. The semiologies are often inconsistent and incoherent, leading to variability and potential limitations in presurgical evaluation. To overcome these challenges, advanced technologies like large language models (LLMs)—with ChatGPT being a notable example—offer valuable tools for analyzing complex textual information, making them well-suited to interpret detailed seizure semiology descriptions and accurately localize the EZ. ObjectiveThis study evaluates the clinical value of ChatGPT for interpreting seizure semiology to localize EZs in presurgical assessments for patients with focal epilepsy and compares its performance with that of epileptologists. MethodsWe compiled 2 data cohorts: a publicly sourced cohort of 852 semiology-EZ pairs from 193 peer-reviewed journal publications and a private cohort of 184 semiology-EZ pairs collected from Far Eastern Memorial Hospital (FEMH) in Taiwan. ChatGPT was evaluated to predict the most likely EZ locations using 2 prompt methods: zero-shot prompting (ZSP) and few-shot prompting (FSP). To compare the performance of ChatGPT, 8 epileptologists were recruited to participate in an online survey to interpret 100 randomly selected semiology records. The responses from ChatGPT and epileptologists were compared using 3 metrics: regional sensitivity (RSens), weighted sensitivity (WSens), and net positive inference rate (NPIR). ResultsIn the publicly sourced cohort, ChatGPT demonstrated high RSens reliability, achieving 80% to 90% for the frontal and temporal lobes; 20% to 40% for the parietal lobe, occipital lobe, and insular cortex; and only 3% for the cingulate cortex. The WSens, which accounts for biased data distribution, consistently exceeded 67%, while the mean NPIR remained around 0. These evaluation results based on the private FEMH cohort are consistent with those from the publicly sourced cohort. A group t test with 1000 bootstrap samples revealed that ChatGPT-4 significantly outperformed epileptologists in RSens for the most frequently implicated EZs, such as the frontal and temporal lobes (P<.001). Additionally, ChatGPT-4 demonstrated superior overall performance in WSens (P<.001). However, no significant differences were observed between ChatGPT and the epileptologists in NPIR, highlighting comparable performance in this metric. ConclusionsChatGPT demonstrated clinical value as a tool to assist decision-making during epilepsy preoperative workups. With ongoing advancements in LLMs, their reliability and accuracy are anticipated to improve.https://www.jmir.org/2025/1/e69173 |
| spellingShingle | Yaxi Luo Meng Jiao Neel Fotedar Jun-En Ding Ioannis Karakis Vikram R Rao Melissa Asmar Xiaochen Xian Orwa Aboud Yuxin Wen Jack J Lin Fang-Ming Hung Hai Sun Felix Rosenow Feng Liu Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation Journal of Medical Internet Research |
| title | Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation |
| title_full | Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation |
| title_fullStr | Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation |
| title_full_unstemmed | Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation |
| title_short | Clinical Value of ChatGPT for Epilepsy Presurgical Decision-Making: Systematic Evaluation of Seizure Semiology Interpretation |
| title_sort | clinical value of chatgpt for epilepsy presurgical decision making systematic evaluation of seizure semiology interpretation |
| url | https://www.jmir.org/2025/1/e69173 |
| work_keys_str_mv | AT yaxiluo clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT mengjiao clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT neelfotedar clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT junending clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT ioanniskarakis clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT vikramrrao clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT melissaasmar clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT xiaochenxian clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT orwaaboud clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT yuxinwen clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT jackjlin clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT fangminghung clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT haisun clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT felixrosenow clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation AT fengliu clinicalvalueofchatgptforepilepsypresurgicaldecisionmakingsystematicevaluationofseizuresemiologyinterpretation |