A systematic review of the impact of artificial intelligence on educational outcomes in health professions education
Abstract Background Artificial intelligence (AI) has a variety of potential applications in health professions education and assessment; however, measurable educational impacts of AI-based educational strategies on learning outcomes have not been systematically evaluated. Methods A systematic litera...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-01-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-025-06719-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571586406776832 |
|---|---|
| author | Eva Feigerlova Hind Hani Ellie Hothersall-Davies |
| author_facet | Eva Feigerlova Hind Hani Ellie Hothersall-Davies |
| author_sort | Eva Feigerlova |
| collection | DOAJ |
| description | Abstract Background Artificial intelligence (AI) has a variety of potential applications in health professions education and assessment; however, measurable educational impacts of AI-based educational strategies on learning outcomes have not been systematically evaluated. Methods A systematic literature search was conducted using electronic databases (CINAHL Plus, EMBASE, Proquest, Pubmed, Cochrane Library, and Web of Science) to identify studies published until October 1st 2024, analyzing the impact of AI-based tools/interventions in health profession assessment and/or training on educational outcomes. The present analysis follows the PRISMA 2020 statement for systematic reviews and the structured approach to reporting in health care education for evidence synthesis. Results The final analysis included twelve studies. All were single centers with sample sizes ranging from 4 to 180 participants. Three studies were randomized controlled trials, and seven had a quasi-experimental design. Two studies were observational. The studies had a heterogenous design. Confounding variables were not controlled. None of the studies provided learning objectives or descriptions of the competencies to be achieved. Three studies applied learning theories in the development of AI-powered educational strategies. One study reported the analysis of the authenticity of the learning environment. No study provided information on the impact of feedback activities on learning outcomes. All studies corresponded to Kirkpatrick’s second level evaluating technical skills or quantifiable knowledge. No study evaluated more complex tasks, such as the behavior of learners in the workplace. There was insufficient information on training datasets and copyright issues. Conclusions The results of the analysis show that the current evidence regarding measurable educational outcomes of AI-powered interventions in health professions education is poor. Further studies with a rigorous methodological approach are needed. The present work also highlights that there is no straightforward guide for evaluating the quality of research in AI-based education and suggests a series of criteria that should be considered. Trial registration Methods and inclusion criteria were defined in advance, specified in a protocol and registered in the OSF registries ( https://osf.io/v5cgp/ ). Clinical Trial number: not applicable. |
| format | Article |
| id | doaj-art-64964af6c65a4979a7dd63d0bb62e678 |
| institution | Kabale University |
| issn | 1472-6920 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj-art-64964af6c65a4979a7dd63d0bb62e6782025-02-02T12:29:56ZengBMCBMC Medical Education1472-69202025-01-0125111810.1186/s12909-025-06719-5A systematic review of the impact of artificial intelligence on educational outcomes in health professions educationEva Feigerlova0Hind Hani1Ellie Hothersall-Davies2Faculté de médecine, maïeutique et métiers de la santé, Université de LorraineFaculté de médecine, maïeutique et métiers de la santé, Université de LorraineCentre for Medical Education, University of DundeeAbstract Background Artificial intelligence (AI) has a variety of potential applications in health professions education and assessment; however, measurable educational impacts of AI-based educational strategies on learning outcomes have not been systematically evaluated. Methods A systematic literature search was conducted using electronic databases (CINAHL Plus, EMBASE, Proquest, Pubmed, Cochrane Library, and Web of Science) to identify studies published until October 1st 2024, analyzing the impact of AI-based tools/interventions in health profession assessment and/or training on educational outcomes. The present analysis follows the PRISMA 2020 statement for systematic reviews and the structured approach to reporting in health care education for evidence synthesis. Results The final analysis included twelve studies. All were single centers with sample sizes ranging from 4 to 180 participants. Three studies were randomized controlled trials, and seven had a quasi-experimental design. Two studies were observational. The studies had a heterogenous design. Confounding variables were not controlled. None of the studies provided learning objectives or descriptions of the competencies to be achieved. Three studies applied learning theories in the development of AI-powered educational strategies. One study reported the analysis of the authenticity of the learning environment. No study provided information on the impact of feedback activities on learning outcomes. All studies corresponded to Kirkpatrick’s second level evaluating technical skills or quantifiable knowledge. No study evaluated more complex tasks, such as the behavior of learners in the workplace. There was insufficient information on training datasets and copyright issues. Conclusions The results of the analysis show that the current evidence regarding measurable educational outcomes of AI-powered interventions in health professions education is poor. Further studies with a rigorous methodological approach are needed. The present work also highlights that there is no straightforward guide for evaluating the quality of research in AI-based education and suggests a series of criteria that should be considered. Trial registration Methods and inclusion criteria were defined in advance, specified in a protocol and registered in the OSF registries ( https://osf.io/v5cgp/ ). Clinical Trial number: not applicable.https://doi.org/10.1186/s12909-025-06719-5Artificial intelligenceHealth professionsAI-based training and assessmentLearning theories and principlesEducational outcomes |
| spellingShingle | Eva Feigerlova Hind Hani Ellie Hothersall-Davies A systematic review of the impact of artificial intelligence on educational outcomes in health professions education BMC Medical Education Artificial intelligence Health professions AI-based training and assessment Learning theories and principles Educational outcomes |
| title | A systematic review of the impact of artificial intelligence on educational outcomes in health professions education |
| title_full | A systematic review of the impact of artificial intelligence on educational outcomes in health professions education |
| title_fullStr | A systematic review of the impact of artificial intelligence on educational outcomes in health professions education |
| title_full_unstemmed | A systematic review of the impact of artificial intelligence on educational outcomes in health professions education |
| title_short | A systematic review of the impact of artificial intelligence on educational outcomes in health professions education |
| title_sort | systematic review of the impact of artificial intelligence on educational outcomes in health professions education |
| topic | Artificial intelligence Health professions AI-based training and assessment Learning theories and principles Educational outcomes |
| url | https://doi.org/10.1186/s12909-025-06719-5 |
| work_keys_str_mv | AT evafeigerlova asystematicreviewoftheimpactofartificialintelligenceoneducationaloutcomesinhealthprofessionseducation AT hindhani asystematicreviewoftheimpactofartificialintelligenceoneducationaloutcomesinhealthprofessionseducation AT elliehothersalldavies asystematicreviewoftheimpactofartificialintelligenceoneducationaloutcomesinhealthprofessionseducation AT evafeigerlova systematicreviewoftheimpactofartificialintelligenceoneducationaloutcomesinhealthprofessionseducation AT hindhani systematicreviewoftheimpactofartificialintelligenceoneducationaloutcomesinhealthprofessionseducation AT elliehothersalldavies systematicreviewoftheimpactofartificialintelligenceoneducationaloutcomesinhealthprofessionseducation |