Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study
Introduction: A growing area is the use of ChatGPT in simulation-based learning, a widely recognized methodology in medical education. This study aimed to evaluate ChatGPT’s ability to generate realistic simulation scenarios to assist faculty as a significant challenge in medical education. Metho...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Shahid Beheshti University of Medical Sciences
2025-05-01
|
| Series: | Archives of Academic Emergency Medicine |
| Subjects: | |
| Online Access: | https://journals.sbmu.ac.ir/aaem/index.php/AAEM/article/view/2690 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850255688740110336 |
|---|---|
| author | Faezeh Ghaffari Mostafa Langarizadeh Ehsan Nabovati Mahdieh Sabery |
| author_facet | Faezeh Ghaffari Mostafa Langarizadeh Ehsan Nabovati Mahdieh Sabery |
| author_sort | Faezeh Ghaffari |
| collection | DOAJ |
| description |
Introduction: A growing area is the use of ChatGPT in simulation-based learning, a widely recognized methodology in medical education. This study aimed to evaluate ChatGPT’s ability to generate realistic simulation scenarios to assist faculty as a significant challenge in medical education.
Method: This study employs a qualitative research design and thematic analysis to interpret expert opinions. The study was conducted in two phases. Scenario generation via ChatGPT and expert review for validation. We used ChatGPT (GPT-4) to create clinical scenarios on cardiovascular topics, including cardiogenic shock, postoperative cardiac tamponade after heart surgery, and heart failure. A panel of five experts, four nurses with expertise in emergency medicine and critical care and an anesthesia specialist, evaluated the scenarios. The experts' feedback, strengths and weaknesses, and proposed revisions from the expert discussions were analyzed via thematic analysis. Key themes and proposed revisions were identified, recorded, and compiled by the research team.
Results: The clinical scenarios were produced by ChatGPT in less than 5 seconds per case. The thematic analysis identified six recurring themes in the experts' discussions: clinical accuracy, the clarity of learning objectives, the logical flow of patient cases, realism and feasibility, alignment with nursing competencies, and level of difficulty. All the experts agreed that the scenarios were realistic and followed clinical guidelines. However, they also identified several errors and areas that needed improvement. The experts identified and documented specific errors, incorrect recommendations, missing information, and inconsistencies with standard nursing practices.
Conclusion: It seems that, ChatGPT can be a valuable tool for developing clinical scenarios, but expert review and refinement are necessary to ensure the accuracy and alignment of the generated scenarios with clinical and educational standards.
|
| format | Article |
| id | doaj-art-5ee116fd43984236a14e7612dc23fae4 |
| institution | OA Journals |
| issn | 2645-4904 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Shahid Beheshti University of Medical Sciences |
| record_format | Article |
| series | Archives of Academic Emergency Medicine |
| spelling | doaj-art-5ee116fd43984236a14e7612dc23fae42025-08-20T01:56:48ZengShahid Beheshti University of Medical SciencesArchives of Academic Emergency Medicine2645-49042025-05-0113110.22037/aaemj.v13i1.2690Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative StudyFaezeh Ghaffari0Mostafa Langarizadeh1Ehsan Nabovati 2Mahdieh Sabery 34. Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran.department of health information management, School of health management and information sciences, Iran University of Medical Sciences, Tehran-IranHealth Information Management Research Center, Kashan University of Medical Sciences, Kashan, IranTrauma Nursing Research Center, Faculty of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran Introduction: A growing area is the use of ChatGPT in simulation-based learning, a widely recognized methodology in medical education. This study aimed to evaluate ChatGPT’s ability to generate realistic simulation scenarios to assist faculty as a significant challenge in medical education. Method: This study employs a qualitative research design and thematic analysis to interpret expert opinions. The study was conducted in two phases. Scenario generation via ChatGPT and expert review for validation. We used ChatGPT (GPT-4) to create clinical scenarios on cardiovascular topics, including cardiogenic shock, postoperative cardiac tamponade after heart surgery, and heart failure. A panel of five experts, four nurses with expertise in emergency medicine and critical care and an anesthesia specialist, evaluated the scenarios. The experts' feedback, strengths and weaknesses, and proposed revisions from the expert discussions were analyzed via thematic analysis. Key themes and proposed revisions were identified, recorded, and compiled by the research team. Results: The clinical scenarios were produced by ChatGPT in less than 5 seconds per case. The thematic analysis identified six recurring themes in the experts' discussions: clinical accuracy, the clarity of learning objectives, the logical flow of patient cases, realism and feasibility, alignment with nursing competencies, and level of difficulty. All the experts agreed that the scenarios were realistic and followed clinical guidelines. However, they also identified several errors and areas that needed improvement. The experts identified and documented specific errors, incorrect recommendations, missing information, and inconsistencies with standard nursing practices. Conclusion: It seems that, ChatGPT can be a valuable tool for developing clinical scenarios, but expert review and refinement are necessary to ensure the accuracy and alignment of the generated scenarios with clinical and educational standards. https://journals.sbmu.ac.ir/aaem/index.php/AAEM/article/view/2690Generative artificial intelligenceArtificial intelligenceEducation, medicalComputer simulation |
| spellingShingle | Faezeh Ghaffari Mostafa Langarizadeh Ehsan Nabovati Mahdieh Sabery Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study Archives of Academic Emergency Medicine Generative artificial intelligence Artificial intelligence Education, medical Computer simulation |
| title | Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study |
| title_full | Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study |
| title_fullStr | Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study |
| title_full_unstemmed | Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study |
| title_short | Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study |
| title_sort | effectiveness of chatgpt for clinical scenario generation a qualitative study |
| topic | Generative artificial intelligence Artificial intelligence Education, medical Computer simulation |
| url | https://journals.sbmu.ac.ir/aaem/index.php/AAEM/article/view/2690 |
| work_keys_str_mv | AT faezehghaffari effectivenessofchatgptforclinicalscenariogenerationaqualitativestudy AT mostafalangarizadeh effectivenessofchatgptforclinicalscenariogenerationaqualitativestudy AT ehsannabovati effectivenessofchatgptforclinicalscenariogenerationaqualitativestudy AT mahdiehsabery effectivenessofchatgptforclinicalscenariogenerationaqualitativestudy |