Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives
Abstract Background The study explores the potential of ChatGPT, an advanced large language model (LLM) by OpenAI, in educating patients about meningioma, a common type of brain tumor. While ChatGPT has generated significant debate regarding its utility and ethics, its growing popularity suggests th...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-06-01
|
| Series: | Radiation Oncology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13014-025-02671-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850218562262663168 |
|---|---|
| author | Diana-Coralia Dehelean Sebastian H. Maier Alev Altay-Langguth Alexander Nitschmann Michael Schmeling Daniel F. Fleischmann Paul Rogowski Christian Trapp Stefanie Corradini Claus Belka Stephan Schönecker Sebastian N. Marschner |
| author_facet | Diana-Coralia Dehelean Sebastian H. Maier Alev Altay-Langguth Alexander Nitschmann Michael Schmeling Daniel F. Fleischmann Paul Rogowski Christian Trapp Stefanie Corradini Claus Belka Stephan Schönecker Sebastian N. Marschner |
| author_sort | Diana-Coralia Dehelean |
| collection | DOAJ |
| description | Abstract Background The study explores the potential of ChatGPT, an advanced large language model (LLM) by OpenAI, in educating patients about meningioma, a common type of brain tumor. While ChatGPT has generated significant debate regarding its utility and ethics, its growing popularity suggests that patients may increasingly use such tools for medical information. The study specifically examines how patients who have undergone radiation therapy for meningioma perceive the information generated by ChatGPT, integrating both patient feedback and clinical assessment. Methods Eight meningioma-related questions on diagnosis, treatment options, and radiation therapy were posed to ChatGPT 4. A questionnaire with these responses and feedback items was developed to assess utility, accuracy, clarity, and alignment with patients’ experiences. Nine clinicians first rated each response’s relevance, correctness, and completeness on a five-point Likert scale. Subsequently, 28 patients with meningioma completed the questionnaire during their first follow-up visit (three months post–radiation therapy). Finally, the same questions were presented to three other large language models (ChatGPT 4o mini, Gemini Free, Gemini Advanced), and seven blinded clinicians rated each model’s responses before selecting the most accurate, eloquent, and comprehensive overall. Results The study cohort included 28 meningioma patients, mostly female, with a median age of 60 years. Most patients found the information clear, accurate, and reflective of their experiences, with 60% willing to use ChatGPT for future inquiries. Clinicians rated the relevance and correctness of the information highly, although completeness was rated slightly lower, particularly for questions about specific radiation therapy details and side effects. ChatGPT 4 and its newer version ChatGPT 4o mini received the highest, nearly identical scores among the four LLMs evaluated, while Gemini Free scored the lowest in clinician assessments. Conclusions ChatGPT demonstrates potential as a supplementary educational tool for meningioma patients, though some areas may require improvement, particularly in providing comprehensive information. The study highlights the potential for integrating AI in patient education, while also noting the need for clinical oversight to ensure accuracy and completeness. Trial registration: LMU ethics vote nr.: 23-0742 |
| format | Article |
| id | doaj-art-ebc5cd5598854d72a7ddef658a2881c6 |
| institution | OA Journals |
| issn | 1748-717X |
| language | English |
| publishDate | 2025-06-01 |
| publisher | BMC |
| record_format | Article |
| series | Radiation Oncology |
| spelling | doaj-art-ebc5cd5598854d72a7ddef658a2881c62025-08-20T02:07:41ZengBMCRadiation Oncology1748-717X2025-06-0120111210.1186/s13014-025-02671-2Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectivesDiana-Coralia Dehelean0Sebastian H. Maier1Alev Altay-Langguth2Alexander Nitschmann3Michael Schmeling4Daniel F. Fleischmann5Paul Rogowski6Christian Trapp7Stefanie Corradini8Claus Belka9Stephan Schönecker10Sebastian N. Marschner11Department of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichDepartment of Radiation Oncology, University Hospital, LMU MunichAbstract Background The study explores the potential of ChatGPT, an advanced large language model (LLM) by OpenAI, in educating patients about meningioma, a common type of brain tumor. While ChatGPT has generated significant debate regarding its utility and ethics, its growing popularity suggests that patients may increasingly use such tools for medical information. The study specifically examines how patients who have undergone radiation therapy for meningioma perceive the information generated by ChatGPT, integrating both patient feedback and clinical assessment. Methods Eight meningioma-related questions on diagnosis, treatment options, and radiation therapy were posed to ChatGPT 4. A questionnaire with these responses and feedback items was developed to assess utility, accuracy, clarity, and alignment with patients’ experiences. Nine clinicians first rated each response’s relevance, correctness, and completeness on a five-point Likert scale. Subsequently, 28 patients with meningioma completed the questionnaire during their first follow-up visit (three months post–radiation therapy). Finally, the same questions were presented to three other large language models (ChatGPT 4o mini, Gemini Free, Gemini Advanced), and seven blinded clinicians rated each model’s responses before selecting the most accurate, eloquent, and comprehensive overall. Results The study cohort included 28 meningioma patients, mostly female, with a median age of 60 years. Most patients found the information clear, accurate, and reflective of their experiences, with 60% willing to use ChatGPT for future inquiries. Clinicians rated the relevance and correctness of the information highly, although completeness was rated slightly lower, particularly for questions about specific radiation therapy details and side effects. ChatGPT 4 and its newer version ChatGPT 4o mini received the highest, nearly identical scores among the four LLMs evaluated, while Gemini Free scored the lowest in clinician assessments. Conclusions ChatGPT demonstrates potential as a supplementary educational tool for meningioma patients, though some areas may require improvement, particularly in providing comprehensive information. The study highlights the potential for integrating AI in patient education, while also noting the need for clinical oversight to ensure accuracy and completeness. Trial registration: LMU ethics vote nr.: 23-0742https://doi.org/10.1186/s13014-025-02671-2MeningiomaChatGPTStereotactic radiosurgeryRadiation therapyPatient experienceLarge language model |
| spellingShingle | Diana-Coralia Dehelean Sebastian H. Maier Alev Altay-Langguth Alexander Nitschmann Michael Schmeling Daniel F. Fleischmann Paul Rogowski Christian Trapp Stefanie Corradini Claus Belka Stephan Schönecker Sebastian N. Marschner Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives Radiation Oncology Meningioma ChatGPT Stereotactic radiosurgery Radiation therapy Patient experience Large language model |
| title | Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives |
| title_full | Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives |
| title_fullStr | Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives |
| title_full_unstemmed | Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives |
| title_short | Evaluating large language models as an educational tool for meningioma patients: patient and clinician perspectives |
| title_sort | evaluating large language models as an educational tool for meningioma patients patient and clinician perspectives |
| topic | Meningioma ChatGPT Stereotactic radiosurgery Radiation therapy Patient experience Large language model |
| url | https://doi.org/10.1186/s13014-025-02671-2 |
| work_keys_str_mv | AT dianacoraliadehelean evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT sebastianhmaier evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT alevaltaylangguth evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT alexandernitschmann evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT michaelschmeling evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT danielffleischmann evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT paulrogowski evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT christiantrapp evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT stefaniecorradini evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT clausbelka evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT stephanschonecker evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives AT sebastiannmarschner evaluatinglargelanguagemodelsasaneducationaltoolformeningiomapatientspatientandclinicianperspectives |