Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania
Objectives To describe a simulation-based rater training curriculum for Objective Structured Clinical Examinations (OSCEs) for clinician-based training for frontline staff caring for mothers and babies in rural Tanzania.Background Rater training for OSCE evaluation is widely embraced in high-income...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2020-02-01
|
| Series: | BMJ Paediatrics Open |
| Online Access: | https://bmjpaedsopen.bmj.com/content/4/1/e000856.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846148538100088832 |
|---|---|
| author | Nalini Singhal Elaine L Sigalet Dismas Matovelo Jennifer L Brenner Maendeleo Boniphace Edgar Ndaboine Lusako Mwaikasu Girles Shabani Julieth Kabirigi Jaelene Mannerfeldt |
| author_facet | Nalini Singhal Elaine L Sigalet Dismas Matovelo Jennifer L Brenner Maendeleo Boniphace Edgar Ndaboine Lusako Mwaikasu Girles Shabani Julieth Kabirigi Jaelene Mannerfeldt |
| author_sort | Nalini Singhal |
| collection | DOAJ |
| description | Objectives To describe a simulation-based rater training curriculum for Objective Structured Clinical Examinations (OSCEs) for clinician-based training for frontline staff caring for mothers and babies in rural Tanzania.Background Rater training for OSCE evaluation is widely embraced in high-income countries but not well described in low-income and middle-income countries. Helping Babies Breathe, Essential Care for Every Baby and Bleeding after Birth are standardised training programmes that encourage OSCE evaluations. Studies examining the reliability of assessments are rare.Methods Training of raters occurred over 3 days. Raters scored selected OSCEs role-played using standardised learners and low-fidelity mannikins, assigning proficiency levels a priori. Researchers used Zabar’s criteria to critique rater agreement and mitigate measurement error during score review. Descriptive statistics, Fleiss’ kappa and field notes were used to describe results.Results Six healthcare providers scored 42 training scenarios. There was moderate rater agreement across all OSCEs (κ=0.508). Kappa values increased with Helping Babies Breathe (κ=0.28–0.48) and Essential Care for Every Baby (κ=0.42–0.77) by day 3 of training, but not with Bleeding after Birth (κ=0.58–0.33). Raters identified average proficiency 50% of the time.Conclusion Our study shows that the in-country raters in this study had a hard time identifying average performance despite moderate rater agreement. Rater training is critical to ensure that the potential of training programmes translates to improved outcomes for mothers and babies; more research into the concepts and training for discernment of competence in this setting is necessary. |
| format | Article |
| id | doaj-art-785e267cd5d345c2a604e77b14d7ffff |
| institution | Kabale University |
| issn | 2399-9772 |
| language | English |
| publishDate | 2020-02-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Paediatrics Open |
| spelling | doaj-art-785e267cd5d345c2a604e77b14d7ffff2024-12-01T03:25:10ZengBMJ Publishing GroupBMJ Paediatrics Open2399-97722020-02-014110.1136/bmjpo-2020-000856Rater training for standardised assessment of Objective Structured Clinical Examinations in rural TanzaniaNalini Singhal0Elaine L Sigalet1Dismas Matovelo2Jennifer L Brenner3Maendeleo Boniphace4Edgar Ndaboine5Lusako Mwaikasu6Girles Shabani7Julieth Kabirigi8Jaelene Mannerfeldt9Pediatrics, University of Calgary, Calgary, Alberta, CanadaCommunity Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, CanadaObstetrics and Gynecology, Nursing, Pediatrics, Catholic University of Health and Allied Sciences, Bugando Medical Center, Mwanza, Mwanza, TanzaniaFaculty of Medicine, University of Calgary Cumming School of Medicine, Calgary, Alberta, CanadaObstetrics and Gynecology, Nursing, Pediatrics, Catholic University of Health and Allied Sciences, Bugando Medical Center, Mwanza, Mwanza, TanzaniaObstetrics and Gynecology, Nursing, Pediatrics, Catholic University of Health and Allied Sciences, Bugando Medical Center, Mwanza, Mwanza, TanzaniaObstetrics and Gynecology, Nursing, Pediatrics, Catholic University of Health and Allied Sciences, Bugando Medical Center, Mwanza, Mwanza, TanzaniaObstetrics and Gynecology, Nursing, Pediatrics, Catholic University of Health and Allied Sciences, Bugando Medical Center, Mwanza, Mwanza, TanzaniaObstetrics and Gynecology, Nursing, Pediatrics, Catholic University of Health and Allied Sciences, Bugando Medical Center, Mwanza, Mwanza, TanzaniaCommunity Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, CanadaObjectives To describe a simulation-based rater training curriculum for Objective Structured Clinical Examinations (OSCEs) for clinician-based training for frontline staff caring for mothers and babies in rural Tanzania.Background Rater training for OSCE evaluation is widely embraced in high-income countries but not well described in low-income and middle-income countries. Helping Babies Breathe, Essential Care for Every Baby and Bleeding after Birth are standardised training programmes that encourage OSCE evaluations. Studies examining the reliability of assessments are rare.Methods Training of raters occurred over 3 days. Raters scored selected OSCEs role-played using standardised learners and low-fidelity mannikins, assigning proficiency levels a priori. Researchers used Zabar’s criteria to critique rater agreement and mitigate measurement error during score review. Descriptive statistics, Fleiss’ kappa and field notes were used to describe results.Results Six healthcare providers scored 42 training scenarios. There was moderate rater agreement across all OSCEs (κ=0.508). Kappa values increased with Helping Babies Breathe (κ=0.28–0.48) and Essential Care for Every Baby (κ=0.42–0.77) by day 3 of training, but not with Bleeding after Birth (κ=0.58–0.33). Raters identified average proficiency 50% of the time.Conclusion Our study shows that the in-country raters in this study had a hard time identifying average performance despite moderate rater agreement. Rater training is critical to ensure that the potential of training programmes translates to improved outcomes for mothers and babies; more research into the concepts and training for discernment of competence in this setting is necessary.https://bmjpaedsopen.bmj.com/content/4/1/e000856.full |
| spellingShingle | Nalini Singhal Elaine L Sigalet Dismas Matovelo Jennifer L Brenner Maendeleo Boniphace Edgar Ndaboine Lusako Mwaikasu Girles Shabani Julieth Kabirigi Jaelene Mannerfeldt Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania BMJ Paediatrics Open |
| title | Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania |
| title_full | Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania |
| title_fullStr | Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania |
| title_full_unstemmed | Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania |
| title_short | Rater training for standardised assessment of Objective Structured Clinical Examinations in rural Tanzania |
| title_sort | rater training for standardised assessment of objective structured clinical examinations in rural tanzania |
| url | https://bmjpaedsopen.bmj.com/content/4/1/e000856.full |
| work_keys_str_mv | AT nalinisinghal ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT elainelsigalet ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT dismasmatovelo ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT jenniferlbrenner ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT maendeleoboniphace ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT edgarndaboine ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT lusakomwaikasu ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT girlesshabani ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT juliethkabirigi ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania AT jaelenemannerfeldt ratertrainingforstandardisedassessmentofobjectivestructuredclinicalexaminationsinruraltanzania |