Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study
Introduction Individual participant data meta-analysis (IPD-MA) is regarded as the gold standard for evidence synthesis. However, diverse recommendations and guidance on its conduct exist, and there is no consensus-based tool for the critical appraisal of a completed IPD-MA. We aim to close this gap...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2025-06-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/15/6/e097297.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849431034942193664 |
|---|---|
| author | Beverley Shea David Moher Mike Clarke Andrea C Tricco Areti Angeliki Veroniki Joel Gagnier Ngianga-bakwin Kandala Edith Ginika Otalike Amanda Doherty-Kirby |
| author_facet | Beverley Shea David Moher Mike Clarke Andrea C Tricco Areti Angeliki Veroniki Joel Gagnier Ngianga-bakwin Kandala Edith Ginika Otalike Amanda Doherty-Kirby |
| author_sort | Beverley Shea |
| collection | DOAJ |
| description | Introduction Individual participant data meta-analysis (IPD-MA) is regarded as the gold standard for evidence synthesis. However, diverse recommendations and guidance on its conduct exist, and there is no consensus-based tool for the critical appraisal of a completed IPD-MA. We aim to close this gap by systematically identifying quality items and developing and validating a critical appraisal checklist for IPD-MA.Methods and analysis This study will comprise three phases, as follows:Phase 1: a systematic methodology review to identify potential checklist domains and items; this will be conducted according to the Cochrane methods for systematic reviews and reported following the Preferred Reporting Items for Systematic Reviews and Meta-analysis 2020 guidance. We will include studies that address methodological guides and essential statistical requirements for IPD-MA. We will use the proposed items to prepare a preliminary checklist for the e-Delphi study.Phase 2: at least two rounds of an e-Delphi survey will be conducted among panels with expertise in IPD-MA research, consensus development, healthcare providers, journal editors, healthcare policymakers, patients and public partners from diverse geographic locations with experience in IPD-MA. Participants will use Qualtrics software to rate items on a 5-point Likert scale. The Wilcoxon matched signed rank test will estimate response stability across rounds. Consensus on including an item will be achieved if ≥75% of the panel rates the item as ‘strongly agree’ or ‘agree’ and items will be excluded if ≥75% rates it as ‘strongly disagree’ or ‘disagree’. A convenience sample of 10 reviewers with experience in conducting an IPD-MA will pilot-test the checklist to provide practical feedback that will be used to refine the checklist.Phase 3: critical appraisal checklist validation: to improve confidence in the tool’s uptake, a subset of the e-Delphi participants and graduate students of epidemiology and biostatistics will conduct content validity and reliability testing, respectively, per the Consensus-based Standards for the Selection of Health Measurement Instruments.Ethics and dissemination Ethics approval has been obtained from the Western University Health Science Research Ethics Board in Canada. The validated checklist will be published in a peer-reviewed open-access journal and shared across the networks of this study’s steering committee, Cochrane IPD-MA group and the institutions’ social media platforms. |
| format | Article |
| id | doaj-art-47693073cb2e40ec9d4dce4a319a020a |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-47693073cb2e40ec9d4dce4a319a020a2025-08-20T03:27:47ZengBMJ Publishing GroupBMJ Open2044-60552025-06-0115610.1136/bmjopen-2024-097297Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi studyBeverley Shea0David Moher1Mike Clarke2Andrea C Tricco3Areti Angeliki Veroniki4Joel Gagnier5Ngianga-bakwin Kandala6Edith Ginika Otalike7Amanda Doherty-Kirby8University of Ottawa, Ottawa, Ontario, CanadaUniversity of Toronto, Toronto, Ontario, CanadaNorthern Ireland Methodology Hub, Queen’s University Belfast, Belfast, UKKnowledge Translation Program, University of Toronto Institute of Health Policy Management and Evaluation, Toronto, Ontario, CanadaIHPME, University of Toronto Institute of Health Policy Management and Evaluation, Toronto, Ontario, CanadaWestern University Schulich School of Medicine & Dentistry, London, Ontario, CanadaEpidemiology and Biostatistics, University of Western Ontario, London, Ontario, CanadaEpidemiology and Biostatistics, University of Western Ontario, London, Ontario, CanadaSt. Michael’s Hospital, Toronto, Ontario, CanadaIntroduction Individual participant data meta-analysis (IPD-MA) is regarded as the gold standard for evidence synthesis. However, diverse recommendations and guidance on its conduct exist, and there is no consensus-based tool for the critical appraisal of a completed IPD-MA. We aim to close this gap by systematically identifying quality items and developing and validating a critical appraisal checklist for IPD-MA.Methods and analysis This study will comprise three phases, as follows:Phase 1: a systematic methodology review to identify potential checklist domains and items; this will be conducted according to the Cochrane methods for systematic reviews and reported following the Preferred Reporting Items for Systematic Reviews and Meta-analysis 2020 guidance. We will include studies that address methodological guides and essential statistical requirements for IPD-MA. We will use the proposed items to prepare a preliminary checklist for the e-Delphi study.Phase 2: at least two rounds of an e-Delphi survey will be conducted among panels with expertise in IPD-MA research, consensus development, healthcare providers, journal editors, healthcare policymakers, patients and public partners from diverse geographic locations with experience in IPD-MA. Participants will use Qualtrics software to rate items on a 5-point Likert scale. The Wilcoxon matched signed rank test will estimate response stability across rounds. Consensus on including an item will be achieved if ≥75% of the panel rates the item as ‘strongly agree’ or ‘agree’ and items will be excluded if ≥75% rates it as ‘strongly disagree’ or ‘disagree’. A convenience sample of 10 reviewers with experience in conducting an IPD-MA will pilot-test the checklist to provide practical feedback that will be used to refine the checklist.Phase 3: critical appraisal checklist validation: to improve confidence in the tool’s uptake, a subset of the e-Delphi participants and graduate students of epidemiology and biostatistics will conduct content validity and reliability testing, respectively, per the Consensus-based Standards for the Selection of Health Measurement Instruments.Ethics and dissemination Ethics approval has been obtained from the Western University Health Science Research Ethics Board in Canada. The validated checklist will be published in a peer-reviewed open-access journal and shared across the networks of this study’s steering committee, Cochrane IPD-MA group and the institutions’ social media platforms.https://bmjopen.bmj.com/content/15/6/e097297.full |
| spellingShingle | Beverley Shea David Moher Mike Clarke Andrea C Tricco Areti Angeliki Veroniki Joel Gagnier Ngianga-bakwin Kandala Edith Ginika Otalike Amanda Doherty-Kirby Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study BMJ Open |
| title | Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study |
| title_full | Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study |
| title_fullStr | Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study |
| title_full_unstemmed | Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study |
| title_short | Development and validation of critical appraisal tool for individual participant data meta-analysis: protocol for a modified e-Delphi study |
| title_sort | development and validation of critical appraisal tool for individual participant data meta analysis protocol for a modified e delphi study |
| url | https://bmjopen.bmj.com/content/15/6/e097297.full |
| work_keys_str_mv | AT beverleyshea developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT davidmoher developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT mikeclarke developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT andreactricco developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT aretiangelikiveroniki developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT joelgagnier developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT ngiangabakwinkandala developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT edithginikaotalike developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy AT amandadohertykirby developmentandvalidationofcriticalappraisaltoolforindividualparticipantdatametaanalysisprotocolforamodifiededelphistudy |