Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group
Abstract BackgroundHealth care practitioners use clinical decision support systems (CDSS) as an aid in the crucial task of clinical reasoning and decision-making. Traditional CDSS are online repositories (ORs) and clinical practice guidelines (CPG). Recently, large language mo...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-03-01
|
| Series: | JMIR Medical Education |
| Online Access: | https://mededu.jmir.org/2025/1/e55709 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850098740594999296 |
|---|---|
| author | Marco Montagna Filippo Chiabrando Rebecca De Lorenzo Patrizia Rovere Querini |
| author_facet | Marco Montagna Filippo Chiabrando Rebecca De Lorenzo Patrizia Rovere Querini |
| author_sort | Marco Montagna |
| collection | DOAJ |
| description |
Abstract
BackgroundHealth care practitioners use clinical decision support systems (CDSS) as an aid in the crucial task of clinical reasoning and decision-making. Traditional CDSS are online repositories (ORs) and clinical practice guidelines (CPG). Recently, large language models (LLMs) such as ChatGPT have emerged as potential alternatives. They have proven to be powerful, innovative tools, yet they are not devoid of worrisome risks.
ObjectiveThis study aims to explore how medical students perform in an evaluated clinical case through the use of different CDSS tools.
MethodsThe authors randomly divided medical students into 3 groups, CPG, n=6 (38%); OR, n=5 (31%); and ChatGPT, n=5 (31%); and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ speed and ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (score: 1‐5). The authors analyzed and categorized group scores according to the skill investigated: differential diagnosis, diagnostic workup, and clinical decision-making.
ResultsAnswering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were as follows: CPG 4.0, OR 3.7, and ChatGPT 3.8 (PP
ConclusionsThis hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in medical degree courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real. |
| format | Article |
| id | doaj-art-75f5237ffd12421abbbd740464d5d650 |
| institution | DOAJ |
| issn | 2369-3762 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Education |
| spelling | doaj-art-75f5237ffd12421abbbd740464d5d6502025-08-20T02:40:39ZengJMIR PublicationsJMIR Medical Education2369-37622025-03-0111e55709e5570910.2196/55709Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus GroupMarco Montagnahttp://orcid.org/0000-0002-0907-7640Filippo Chiabrandohttp://orcid.org/0009-0004-3680-7559Rebecca De Lorenzohttp://orcid.org/0000-0002-1281-7996Patrizia Rovere Querinihttp://orcid.org/0000-0003-2615-3649 Abstract BackgroundHealth care practitioners use clinical decision support systems (CDSS) as an aid in the crucial task of clinical reasoning and decision-making. Traditional CDSS are online repositories (ORs) and clinical practice guidelines (CPG). Recently, large language models (LLMs) such as ChatGPT have emerged as potential alternatives. They have proven to be powerful, innovative tools, yet they are not devoid of worrisome risks. ObjectiveThis study aims to explore how medical students perform in an evaluated clinical case through the use of different CDSS tools. MethodsThe authors randomly divided medical students into 3 groups, CPG, n=6 (38%); OR, n=5 (31%); and ChatGPT, n=5 (31%); and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ speed and ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (score: 1‐5). The authors analyzed and categorized group scores according to the skill investigated: differential diagnosis, diagnostic workup, and clinical decision-making. ResultsAnswering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were as follows: CPG 4.0, OR 3.7, and ChatGPT 3.8 (PP ConclusionsThis hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in medical degree courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real.https://mededu.jmir.org/2025/1/e55709 |
| spellingShingle | Marco Montagna Filippo Chiabrando Rebecca De Lorenzo Patrizia Rovere Querini Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group JMIR Medical Education |
| title | Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group |
| title_full | Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group |
| title_fullStr | Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group |
| title_full_unstemmed | Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group |
| title_short | Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group |
| title_sort | impact of clinical decision support systems on medical students case solving performance comparison study with a focus group |
| url | https://mededu.jmir.org/2025/1/e55709 |
| work_keys_str_mv | AT marcomontagna impactofclinicaldecisionsupportsystemsonmedicalstudentscasesolvingperformancecomparisonstudywithafocusgroup AT filippochiabrando impactofclinicaldecisionsupportsystemsonmedicalstudentscasesolvingperformancecomparisonstudywithafocusgroup AT rebeccadelorenzo impactofclinicaldecisionsupportsystemsonmedicalstudentscasesolvingperformancecomparisonstudywithafocusgroup AT patriziaroverequerini impactofclinicaldecisionsupportsystemsonmedicalstudentscasesolvingperformancecomparisonstudywithafocusgroup |