Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study
Introduction Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners’ judgements which can challenge assessment equivalence and (3) for limited d...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2022-12-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/12/12/e064387.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849324302108721152 |
|---|---|
| author | Geoff Wong Kathy Cullen Rhian Goodfellow Gareth McCray Richard Fuller Robert K McKinley Ruth Kinston Peter Yeates Adriano Maluf Natalie Cope Vikki O’Neill Aidan Cole Rebecca Vallender Ching-Wa Chung |
| author_facet | Geoff Wong Kathy Cullen Rhian Goodfellow Gareth McCray Richard Fuller Robert K McKinley Ruth Kinston Peter Yeates Adriano Maluf Natalie Cope Vikki O’Neill Aidan Cole Rebecca Vallender Ching-Wa Chung |
| author_sort | Geoff Wong |
| collection | DOAJ |
| description | Introduction Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners’ judgements which can challenge assessment equivalence and (3) for limited diagnosticity of trainees’ focal strengths and weaknesses. In response, this study aims to investigate whether (1) sharing integrated-task OSCE stations across institutions can increase perceived authenticity, while (2) enhancing assessment equivalence by enabling comparison of the standard of examiners’ judgements between institutions using a novel methodology (video-based score comparison and adjustment (VESCA)) and (3) exploring the potential to develop more diagnostic signals from data on students’ performances.Methods and analysis The study will use a complex intervention design, developing, implementing and sharing an integrated-task (research) OSCE across four UK medical schools. It will use VESCA to compare examiner scoring differences between groups of examiners and different sites, while studying how, why and for whom the shared OSCE and VESCA operate across participating schools. Quantitative analysis will use Many Facet Rasch Modelling to compare the influence of different examiners groups and sites on students’ scores, while the operation of the two interventions (shared integrated task OSCEs; VESCA) will be studied through the theory-driven method of Realist evaluation. Further exploratory analyses will examine diagnostic performance signals within data.Ethics and dissemination The study will be extra to usual course requirements and all participation will be voluntary. We will uphold principles of informed consent, the right to withdraw, confidentiality with pseudonymity and strict data security. The study has received ethical approval from Keele University Research Ethics Committee. Findings will be academically published and will contribute to good practice guidance on (1) the use of VESCA and (2) sharing and use of integrated-task OSCE stations. |
| format | Article |
| id | doaj-art-3e23a706f6df4d60a0a590bc2aec1e0a |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2022-12-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-3e23a706f6df4d60a0a590bc2aec1e0a2025-08-20T03:48:46ZengBMJ Publishing GroupBMJ Open2044-60552022-12-01121210.1136/bmjopen-2022-064387Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention studyGeoff Wong0Kathy Cullen1Rhian Goodfellow2Gareth McCray3Richard Fuller4Robert K McKinley5Ruth Kinston6Peter Yeates7Adriano Maluf8Natalie Cope9Vikki O’Neill10Aidan Cole11Rebecca Vallender12Ching-Wa Chung13Nuffield Department of Primary Care Health Sciences, Oxford University, Oxford, UK3RespiratorySchool of Medicine, Cardiff University, Cardiff, UKSchool of Medicine, Keele University, Keele, UKSchool of Medicine, University of Liverpool Faculty of Health and Life Sciences, Liverpool, UKemeritus professor of education in general practiceSchool of Medicine, Keele University, Keele, Staffordshire, UKSchool of Medicine, Keele University, Keele, Staffordshire, UKSchool of Medicine, Keele University, Keele, Staffordshire, UKSchool of Medicine, Keele University, Keele, Staffordshire, UKSchool of Medicine, Dentistry and Biomedical Sciences, Queen`s University Belfast, Belfast, UKSchool of Medicine, Dentistry and Biomedical Sciences, Queen`s University Belfast, Belfast, UKSchool of Medicine, Cardiff University, Cardiff, UKSchool of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, Scotland, UKIntroduction Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners’ judgements which can challenge assessment equivalence and (3) for limited diagnosticity of trainees’ focal strengths and weaknesses. In response, this study aims to investigate whether (1) sharing integrated-task OSCE stations across institutions can increase perceived authenticity, while (2) enhancing assessment equivalence by enabling comparison of the standard of examiners’ judgements between institutions using a novel methodology (video-based score comparison and adjustment (VESCA)) and (3) exploring the potential to develop more diagnostic signals from data on students’ performances.Methods and analysis The study will use a complex intervention design, developing, implementing and sharing an integrated-task (research) OSCE across four UK medical schools. It will use VESCA to compare examiner scoring differences between groups of examiners and different sites, while studying how, why and for whom the shared OSCE and VESCA operate across participating schools. Quantitative analysis will use Many Facet Rasch Modelling to compare the influence of different examiners groups and sites on students’ scores, while the operation of the two interventions (shared integrated task OSCEs; VESCA) will be studied through the theory-driven method of Realist evaluation. Further exploratory analyses will examine diagnostic performance signals within data.Ethics and dissemination The study will be extra to usual course requirements and all participation will be voluntary. We will uphold principles of informed consent, the right to withdraw, confidentiality with pseudonymity and strict data security. The study has received ethical approval from Keele University Research Ethics Committee. Findings will be academically published and will contribute to good practice guidance on (1) the use of VESCA and (2) sharing and use of integrated-task OSCE stations.https://bmjopen.bmj.com/content/12/12/e064387.full |
| spellingShingle | Geoff Wong Kathy Cullen Rhian Goodfellow Gareth McCray Richard Fuller Robert K McKinley Ruth Kinston Peter Yeates Adriano Maluf Natalie Cope Vikki O’Neill Aidan Cole Rebecca Vallender Ching-Wa Chung Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study BMJ Open |
| title | Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study |
| title_full | Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study |
| title_fullStr | Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study |
| title_full_unstemmed | Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study |
| title_short | Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study |
| title_sort | enhancing authenticity diagnosticity and equivalence ad equiv in multicentre osce exams in health professionals education protocol for a complex intervention study |
| url | https://bmjopen.bmj.com/content/12/12/e064387.full |
| work_keys_str_mv | AT geoffwong enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT kathycullen enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT rhiangoodfellow enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT garethmccray enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT richardfuller enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT robertkmckinley enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT ruthkinston enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT peteryeates enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT adrianomaluf enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT nataliecope enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT vikkioneill enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT aidancole enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT rebeccavallender enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy AT chingwachung enhancingauthenticitydiagnosticityandequivalenceadequivinmulticentreosceexamsinhealthprofessionalseducationprotocolforacomplexinterventionstudy |