Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries
Introduction Methods and validated tools for evaluating the performance of competency-based implementation research (IR) training programmes in low–middle-income countries (LMICs) are lacking. In this study, we developed tools for assessing the performance of IR training programmes based on a framew...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2024-07-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/14/7/e082250.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850067023433826304 |
|---|---|
| author | Latifat Ibisomi Olakunle Alonge Aditi Rao Anna Kalbarczyk Edwin Rolando Gonzalez Marulanda Phyllis Dako-Gyeke Yodi Mahendradhata Pascal Launois Mahnaz Vahedi Choolwe Jacobs Kwasi Torpey Carlos Alberto Rojas |
| author_facet | Latifat Ibisomi Olakunle Alonge Aditi Rao Anna Kalbarczyk Edwin Rolando Gonzalez Marulanda Phyllis Dako-Gyeke Yodi Mahendradhata Pascal Launois Mahnaz Vahedi Choolwe Jacobs Kwasi Torpey Carlos Alberto Rojas |
| author_sort | Latifat Ibisomi |
| collection | DOAJ |
| description | Introduction Methods and validated tools for evaluating the performance of competency-based implementation research (IR) training programmes in low–middle-income countries (LMICs) are lacking. In this study, we developed tools for assessing the performance of IR training programmes based on a framework of IR core competencies in LMICs.Methods We developed self-assessment and objective-assessment tools drawing on the IR competency framework. We used exploratory factor analyses and a one-parameter logistic model to establish construct validity and internal consistency of the tools drawing on a survey conducted in 2020 with 166 trainees before and after an index IR course across five universities and LMICs under the Special Program for Research and Training in Tropical Diseases postgraduate IR training scheme. We conducted key informant interviews (KII) with 20 trainees and five trainers to reflect on the usefulness of the tools and framework for guiding IR training in LMICs.Results Two 16-item tools for self-assessment of IR knowledge and self-efficacy and a 40-item objective assessment tool were developed. The factor loadings of items in the self-assessment tools were 0.65–0.87 with Cronbach’s alpha (α) of 0.97, and 0.77–0.91 with α of 0.98 for the IR knowledge and self-efficacy tools, respectively. The distribution of item difficulty in the objective-assessment tool was consistent before and after the index IR course. Pearson correlation coefficient (r) between the self-assessed and objectively assessed IR knowledge before the index IR course was low, r=0.27 (p value: <0.01), with slight improvements after the index IR course, r=0.43 (p value: <0.01). All KII respondents reported the assessment tools and framework were valid for assessing IR competencies.Conclusion The IR competency framework and tools developed for assessing IR competencies and performance of IR training programmes in LMICs are reliable and valid. Self-assessment methods alone may not suffice to yield a valid assessment of performance in these settings. |
| format | Article |
| id | doaj-art-ef33c2f5311b4d7289f2d3075fbc9f15 |
| institution | DOAJ |
| issn | 2044-6055 |
| language | English |
| publishDate | 2024-07-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-ef33c2f5311b4d7289f2d3075fbc9f152025-08-20T02:48:30ZengBMJ Publishing GroupBMJ Open2044-60552024-07-0114710.1136/bmjopen-2023-082250Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countriesLatifat Ibisomi0Olakunle Alonge1Aditi Rao2Anna Kalbarczyk3Edwin Rolando Gonzalez Marulanda4Phyllis Dako-Gyeke5Yodi Mahendradhata6Pascal Launois7Mahnaz Vahedi8Choolwe Jacobs9Kwasi Torpey10Carlos Alberto Rojas11Public Health and Epidemiology, Nigerian Institute of Medical Research (NIMR), Lagos, Nigeria1 Sparkman Center for Global Health, The University of Alabama at Birmingham, Birmingham, Alabama, USAInternational Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, USA1 Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA6 Facultad Nacional de Salud Publica, Universidad de Antioquia, Medellin, Colombia4 Social and Behavioural Sciences, University of Ghana School of Public Health, Accra, GhanaDepartment of Health Policy and Management, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia9 Special Programme for Research & Training in Tropical Diseases (TDR), WHO, Geneva, Switzerland1 UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR), WHO, Geneva, SwitzerlandDepartment of Epidemiology and Biostatistics, University of Zambia, Lusaka, Zambia8 University of Ghana, Legon, Greater Accra, Ghana6 Facultad Nacional de Salud Publica, Universidad de Antioquia, Medellin, ColombiaIntroduction Methods and validated tools for evaluating the performance of competency-based implementation research (IR) training programmes in low–middle-income countries (LMICs) are lacking. In this study, we developed tools for assessing the performance of IR training programmes based on a framework of IR core competencies in LMICs.Methods We developed self-assessment and objective-assessment tools drawing on the IR competency framework. We used exploratory factor analyses and a one-parameter logistic model to establish construct validity and internal consistency of the tools drawing on a survey conducted in 2020 with 166 trainees before and after an index IR course across five universities and LMICs under the Special Program for Research and Training in Tropical Diseases postgraduate IR training scheme. We conducted key informant interviews (KII) with 20 trainees and five trainers to reflect on the usefulness of the tools and framework for guiding IR training in LMICs.Results Two 16-item tools for self-assessment of IR knowledge and self-efficacy and a 40-item objective assessment tool were developed. The factor loadings of items in the self-assessment tools were 0.65–0.87 with Cronbach’s alpha (α) of 0.97, and 0.77–0.91 with α of 0.98 for the IR knowledge and self-efficacy tools, respectively. The distribution of item difficulty in the objective-assessment tool was consistent before and after the index IR course. Pearson correlation coefficient (r) between the self-assessed and objectively assessed IR knowledge before the index IR course was low, r=0.27 (p value: <0.01), with slight improvements after the index IR course, r=0.43 (p value: <0.01). All KII respondents reported the assessment tools and framework were valid for assessing IR competencies.Conclusion The IR competency framework and tools developed for assessing IR competencies and performance of IR training programmes in LMICs are reliable and valid. Self-assessment methods alone may not suffice to yield a valid assessment of performance in these settings.https://bmjopen.bmj.com/content/14/7/e082250.full |
| spellingShingle | Latifat Ibisomi Olakunle Alonge Aditi Rao Anna Kalbarczyk Edwin Rolando Gonzalez Marulanda Phyllis Dako-Gyeke Yodi Mahendradhata Pascal Launois Mahnaz Vahedi Choolwe Jacobs Kwasi Torpey Carlos Alberto Rojas Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries BMJ Open |
| title | Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries |
| title_full | Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries |
| title_fullStr | Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries |
| title_full_unstemmed | Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries |
| title_short | Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries |
| title_sort | multimethods study to develop tools for competency based assessments of implementation research training programmes in low and middle income countries |
| url | https://bmjopen.bmj.com/content/14/7/e082250.full |
| work_keys_str_mv | AT latifatibisomi multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT olakunlealonge multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT aditirao multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT annakalbarczyk multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT edwinrolandogonzalezmarulanda multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT phyllisdakogyeke multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT yodimahendradhata multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT pascallaunois multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT mahnazvahedi multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT choolwejacobs multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT kwasitorpey multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries AT carlosalbertorojas multimethodsstudytodeveloptoolsforcompetencybasedassessmentsofimplementationresearchtrainingprogrammesinlowandmiddleincomecountries |