Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care
The COVID-19 pandemic triggered transformations in academic medicine, rapidly adopting remote teaching and online assessments. Whilst virtual environments show promise in evaluating medical knowledge, their impact on examiner workload is unclear. This study explores examiner’s workload during differ...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2024-12-01
|
| Series: | Medical Education Online |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/10872981.2024.2364990 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850065081718538240 |
|---|---|
| author | Mikhail Dziadzko Andrey Varvinskiy Rodolphe Di Loreto Hugues Scipioni Bazil Ateleanu Markus Klimek Joana Berger-Estilita |
| author_facet | Mikhail Dziadzko Andrey Varvinskiy Rodolphe Di Loreto Hugues Scipioni Bazil Ateleanu Markus Klimek Joana Berger-Estilita |
| author_sort | Mikhail Dziadzko |
| collection | DOAJ |
| description | The COVID-19 pandemic triggered transformations in academic medicine, rapidly adopting remote teaching and online assessments. Whilst virtual environments show promise in evaluating medical knowledge, their impact on examiner workload is unclear. This study explores examiner’s workload during different European Diploma in Anaesthesiology and Intensive Care Part 2 Structured Oral Examinations formats. We hypothesise that online exams result in lower examiner’s workload than traditional face-to-face methods. We also investigate workload structure and its correlation with examiner characteristics and marking performance. In 2023, examiner’s workload for three examination formats (face-to-face, hybrid, online) using the NASA TLX instrument was prospectively evaluated. The impact of examiner demographics, candidate scoring agreement, and examination scores on workload was analysed. The overall NASA TLX score from 215 workload measurements in 142 examiners was high at 59.61 ± 14.13. The online examination had a statistically higher workload (61.65 ± 12.84) than hybrid but not face-to-face. Primary contributors to workload were mental and temporal demands, and effort. Online exams were associated with elevated frustration. Male examiners and those spending more time on exam preparation experienced a higher workload. Multiple diploma specialties and familiarity with European Diploma in Anaesthesiology and Intensive Care exams were protective against high workload. Perceived workload did not impact marking agreement or examination scores across all formats. Examiners experience high workload. Online exams are not systematically associated with decreased workload, likely due to frustration. Despite workload differences, no impact on examiner’s performance or examination scores was found. The hybrid examination mode, combining face-to-face and online, was associated with a minor but statistically significant workload reduction. This hybrid approach may offer a more balanced and efficient examination process while maintaining integrity, cost savings, and increased accessibility for candidates. |
| format | Article |
| id | doaj-art-e0216f18bb014fea95722344ad7489b9 |
| institution | DOAJ |
| issn | 1087-2981 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Medical Education Online |
| spelling | doaj-art-e0216f18bb014fea95722344ad7489b92025-08-20T02:49:06ZengTaylor & Francis GroupMedical Education Online1087-29812024-12-0129110.1080/10872981.2024.2364990Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive careMikhail Dziadzko0Andrey Varvinskiy1Rodolphe Di Loreto2Hugues Scipioni3Bazil Ateleanu4Markus Klimek5Joana Berger-Estilita6Department of Anesthesia, Intensive Care and Pain Management, Hospices Civils de Lyon, Hôpital de la Croix Rousse, Lyon, FranceSouth Devon Healthcare NHS Foundation Trust, Department of Anesthesia and Intensive Care, Torquay, UKEuropean Society of Anaesthesiology and Intensive Care, Examinations Office, Brussels, BelgiumEuropean Society of Anaesthesiology and Intensive Care, Examinations Office, Brussels, BelgiumEuropean Society of Anesthesiology and Intensive Care, Examinations Committee, Brussels, BelgiumEuropean Society of Anesthesiology and Intensive Care, Examinations Committee, Brussels, BelgiumEuropean Society of Anesthesiology and Intensive Care, Examinations Committee, Brussels, BelgiumThe COVID-19 pandemic triggered transformations in academic medicine, rapidly adopting remote teaching and online assessments. Whilst virtual environments show promise in evaluating medical knowledge, their impact on examiner workload is unclear. This study explores examiner’s workload during different European Diploma in Anaesthesiology and Intensive Care Part 2 Structured Oral Examinations formats. We hypothesise that online exams result in lower examiner’s workload than traditional face-to-face methods. We also investigate workload structure and its correlation with examiner characteristics and marking performance. In 2023, examiner’s workload for three examination formats (face-to-face, hybrid, online) using the NASA TLX instrument was prospectively evaluated. The impact of examiner demographics, candidate scoring agreement, and examination scores on workload was analysed. The overall NASA TLX score from 215 workload measurements in 142 examiners was high at 59.61 ± 14.13. The online examination had a statistically higher workload (61.65 ± 12.84) than hybrid but not face-to-face. Primary contributors to workload were mental and temporal demands, and effort. Online exams were associated with elevated frustration. Male examiners and those spending more time on exam preparation experienced a higher workload. Multiple diploma specialties and familiarity with European Diploma in Anaesthesiology and Intensive Care exams were protective against high workload. Perceived workload did not impact marking agreement or examination scores across all formats. Examiners experience high workload. Online exams are not systematically associated with decreased workload, likely due to frustration. Despite workload differences, no impact on examiner’s performance or examination scores was found. The hybrid examination mode, combining face-to-face and online, was associated with a minor but statistically significant workload reduction. This hybrid approach may offer a more balanced and efficient examination process while maintaining integrity, cost savings, and increased accessibility for candidates.https://www.tandfonline.com/doi/10.1080/10872981.2024.2364990Anaesthesiologycontinued educationevaluationexaminer taskloadonline assessmentstructured oral examination |
| spellingShingle | Mikhail Dziadzko Andrey Varvinskiy Rodolphe Di Loreto Hugues Scipioni Bazil Ateleanu Markus Klimek Joana Berger-Estilita Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care Medical Education Online Anaesthesiology continued education evaluation examiner taskload online assessment structured oral examination |
| title | Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care |
| title_full | Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care |
| title_fullStr | Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care |
| title_full_unstemmed | Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care |
| title_short | Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care |
| title_sort | examiner workload comparison three structured oral examination formats for the european diploma in anaesthesiology and intensive care |
| topic | Anaesthesiology continued education evaluation examiner taskload online assessment structured oral examination |
| url | https://www.tandfonline.com/doi/10.1080/10872981.2024.2364990 |
| work_keys_str_mv | AT mikhaildziadzko examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare AT andreyvarvinskiy examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare AT rodolphediloreto examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare AT huguesscipioni examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare AT bazilateleanu examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare AT markusklimek examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare AT joanabergerestilita examinerworkloadcomparisonthreestructuredoralexaminationformatsfortheeuropeandiplomainanaesthesiologyandintensivecare |