Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review
Background When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potentia...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2020-10-01
|
| Series: | BMJ Global Health |
| Online Access: | https://gh.bmj.com/content/5/10/e003567.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850247289837191168 |
|---|---|
| author | Larry D Lynd Mohammad Karamouzian Michael R Law Hinda Ruton Celestin Hategeka |
| author_facet | Larry D Lynd Mohammad Karamouzian Michael R Law Hinda Ruton Celestin Hategeka |
| author_sort | Larry D Lynd |
| collection | DOAJ |
| description | Background When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions.Methods We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427).Results Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time.Conclusions The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions. |
| format | Article |
| id | doaj-art-00cdf382f7194bc1becb2c8696192daa |
| institution | OA Journals |
| issn | 2059-7908 |
| language | English |
| publishDate | 2020-10-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Global Health |
| spelling | doaj-art-00cdf382f7194bc1becb2c8696192daa2025-08-20T01:58:59ZengBMJ Publishing GroupBMJ Global Health2059-79082020-10-0151010.1136/bmjgh-2020-003567Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic reviewLarry D Lynd0Mohammad Karamouzian1Michael R Law2Hinda Ruton3Celestin Hategeka4Center for Health Evaluation and Outcome Sciences, Providence Health Research Institute, Vancouver, British Columbia, Canada2 HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, IranCentre for Health Services and Policy Research, School of Population and Public Health, The University of British Columbia, Vancouver, British Columbia, Canada2 Centre for Health Services and Policy Research, The University of British Columbia, Vancouver, British Columbia, CanadaDepartment of Global Health and Population, Harvard TH Chan School of Public Health, Boston, Massachusetts, USABackground When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions.Methods We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427).Results Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time.Conclusions The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.https://gh.bmj.com/content/5/10/e003567.full |
| spellingShingle | Larry D Lynd Mohammad Karamouzian Michael R Law Hinda Ruton Celestin Hategeka Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review BMJ Global Health |
| title | Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review |
| title_full | Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review |
| title_fullStr | Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review |
| title_full_unstemmed | Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review |
| title_short | Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review |
| title_sort | use of interrupted time series methods in the evaluation of health system quality improvement interventions a methodological systematic review |
| url | https://gh.bmj.com/content/5/10/e003567.full |
| work_keys_str_mv | AT larrydlynd useofinterruptedtimeseriesmethodsintheevaluationofhealthsystemqualityimprovementinterventionsamethodologicalsystematicreview AT mohammadkaramouzian useofinterruptedtimeseriesmethodsintheevaluationofhealthsystemqualityimprovementinterventionsamethodologicalsystematicreview AT michaelrlaw useofinterruptedtimeseriesmethodsintheevaluationofhealthsystemqualityimprovementinterventionsamethodologicalsystematicreview AT hindaruton useofinterruptedtimeseriesmethodsintheevaluationofhealthsystemqualityimprovementinterventionsamethodologicalsystematicreview AT celestinhategeka useofinterruptedtimeseriesmethodsintheevaluationofhealthsystemqualityimprovementinterventionsamethodologicalsystematicreview |