A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study
Abstract Background The general medicine in-training examination (GM-ITE) assesses physicians’ clinical knowledge. This study expanded on findings from a previous pilot study to assess the relationship between general medicine in-training examination (GM-ITE) scores and the diagnostic skills of resi...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-11-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-024-06395-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850216365848264704 |
|---|---|
| author | Kiyoshi Shikino Yuji Nishizaki Koshi Kataoka Sho Fukui Daiki Yokokawa Taro Shimizu Yu Yamamoto Kazuya Nagasaki Hiroyuki Kobayashi Yasuharu Tokuda |
| author_facet | Kiyoshi Shikino Yuji Nishizaki Koshi Kataoka Sho Fukui Daiki Yokokawa Taro Shimizu Yu Yamamoto Kazuya Nagasaki Hiroyuki Kobayashi Yasuharu Tokuda |
| author_sort | Kiyoshi Shikino |
| collection | DOAJ |
| description | Abstract Background The general medicine in-training examination (GM-ITE) assesses physicians’ clinical knowledge. This study expanded on findings from a previous pilot study to assess the relationship between general medicine in-training examination (GM-ITE) scores and the diagnostic skills of resident physicians in Japan by employing an innovative clinical simulation video (CSV-IE). Methods This multicenter cross-sectional study included 4,677 resident physicians who took the GMITE between January 17 and 30, 2023. Participants watched the CSV-IE, depicting an emergency room scenario, and provided a diagnosis. The CSV-IE depicts an emergency case and provides a diagnosis. Discrimination indices were used to assess the CSV-IE’s effectiveness across clinical competence domains, and multilevel logistic regression was used to analyze physician- and hospital-level factors associated with correct diagnoses. Results Correct diagnoses were provided by 470 participants (10.0%). The CSV-IE demonstrated high discriminatory power across all assessed domains, including basic clinical knowledge (DI = 0.44), symptomatology and clinical reasoning (DI = 0.31), physical examination and clinical procedure (DI = 0.35), and knowledge about the disease (DI = 0.25), supporting its utility as an effective assessment tool. In the multivariable analysis, factors associated with a higher likelihood of providing a correct CSV-IE diagnosis included a higher annual number of emergency outpatients (adjusted odds ratio: 1.025; 95% confidence interval [CI]: 1.003–1.047; P = .0230) and being in a higher postgraduate year (adjusted odds ratio: 1.387; 95% CI: 1.104–1.742; P = .005). Conversely, resident physicians at university hospitals were less likely to provide a correct CSV-IE response (adjusted odds ratio: 0.624; 95% CI: 0.435–0.896; P = .0107). Conclusions CSV-IE modules may provide an integrative and realistic evaluation of clinical competence, addressing limitations of traditional MCQ-based assessments by offering contextualized, real-world scenarios that require dynamic decision-making and diagnostic reasoning. |
| format | Article |
| id | doaj-art-bff6483be1d148d2993a76c4dd2ff832 |
| institution | OA Journals |
| issn | 1472-6920 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj-art-bff6483be1d148d2993a76c4dd2ff8322025-08-20T02:08:19ZengBMCBMC Medical Education1472-69202024-11-0124111010.1186/s12909-024-06395-xA novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional studyKiyoshi Shikino0Yuji Nishizaki1Koshi Kataoka2Sho Fukui3Daiki Yokokawa4Taro Shimizu5Yu Yamamoto6Kazuya Nagasaki7Hiroyuki Kobayashi8Yasuharu Tokuda9Department of Community-Oriented Medical Education, Chiba University Graduate School of MedicineDivision of Medical Education, Juntendo University School of MedicineDivision of Medical Education, Juntendo University School of MedicineDepartment of General Medicine, Kyorin University School of MedicineDepartment of General Medicine, Chiba University HospitalDepartment of Diagnostic and Generalist Medicine, Dokkyo Medical University HospitalDivision of General Medicine, Center for Community Medicine, Jichi Medical UniversityUniversity of TsukubaUniversity of TsukubaMuribushi Okinawa Center for Teaching HospitalsAbstract Background The general medicine in-training examination (GM-ITE) assesses physicians’ clinical knowledge. This study expanded on findings from a previous pilot study to assess the relationship between general medicine in-training examination (GM-ITE) scores and the diagnostic skills of resident physicians in Japan by employing an innovative clinical simulation video (CSV-IE). Methods This multicenter cross-sectional study included 4,677 resident physicians who took the GMITE between January 17 and 30, 2023. Participants watched the CSV-IE, depicting an emergency room scenario, and provided a diagnosis. The CSV-IE depicts an emergency case and provides a diagnosis. Discrimination indices were used to assess the CSV-IE’s effectiveness across clinical competence domains, and multilevel logistic regression was used to analyze physician- and hospital-level factors associated with correct diagnoses. Results Correct diagnoses were provided by 470 participants (10.0%). The CSV-IE demonstrated high discriminatory power across all assessed domains, including basic clinical knowledge (DI = 0.44), symptomatology and clinical reasoning (DI = 0.31), physical examination and clinical procedure (DI = 0.35), and knowledge about the disease (DI = 0.25), supporting its utility as an effective assessment tool. In the multivariable analysis, factors associated with a higher likelihood of providing a correct CSV-IE diagnosis included a higher annual number of emergency outpatients (adjusted odds ratio: 1.025; 95% confidence interval [CI]: 1.003–1.047; P = .0230) and being in a higher postgraduate year (adjusted odds ratio: 1.387; 95% CI: 1.104–1.742; P = .005). Conversely, resident physicians at university hospitals were less likely to provide a correct CSV-IE response (adjusted odds ratio: 0.624; 95% CI: 0.435–0.896; P = .0107). Conclusions CSV-IE modules may provide an integrative and realistic evaluation of clinical competence, addressing limitations of traditional MCQ-based assessments by offering contextualized, real-world scenarios that require dynamic decision-making and diagnostic reasoning.https://doi.org/10.1186/s12909-024-06395-xAssessmentClinical competenceClinical simulation videoGeneral medicine in-training examinationPostgraduate medical education |
| spellingShingle | Kiyoshi Shikino Yuji Nishizaki Koshi Kataoka Sho Fukui Daiki Yokokawa Taro Shimizu Yu Yamamoto Kazuya Nagasaki Hiroyuki Kobayashi Yasuharu Tokuda A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study BMC Medical Education Assessment Clinical competence Clinical simulation video General medicine in-training examination Postgraduate medical education |
| title | A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study |
| title_full | A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study |
| title_fullStr | A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study |
| title_full_unstemmed | A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study |
| title_short | A novel resident physician examination using clinical simulation video to assess clinical competence in Japan: a cross-sectional study |
| title_sort | novel resident physician examination using clinical simulation video to assess clinical competence in japan a cross sectional study |
| topic | Assessment Clinical competence Clinical simulation video General medicine in-training examination Postgraduate medical education |
| url | https://doi.org/10.1186/s12909-024-06395-x |
| work_keys_str_mv | AT kiyoshishikino anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT yujinishizaki anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT koshikataoka anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT shofukui anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT daikiyokokawa anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT taroshimizu anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT yuyamamoto anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT kazuyanagasaki anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT hiroyukikobayashi anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT yasuharutokuda anovelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT kiyoshishikino novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT yujinishizaki novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT koshikataoka novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT shofukui novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT daikiyokokawa novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT taroshimizu novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT yuyamamoto novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT kazuyanagasaki novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT hiroyukikobayashi novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy AT yasuharutokuda novelresidentphysicianexaminationusingclinicalsimulationvideotoassessclinicalcompetenceinjapanacrosssectionalstudy |