Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data
Objective To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longi...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2024-02-01
|
| Series: | BMJ Medicine |
| Online Access: | https://bmjmedicine.bmj.com/content/3/1/e000709.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850203641945784320 |
|---|---|
| author | Sebastian Schneeweiss Shirley V Wang Rachel Heyard Leonhard Held |
| author_facet | Sebastian Schneeweiss Shirley V Wang Rachel Heyard Leonhard Held |
| author_sort | Sebastian Schneeweiss |
| collection | DOAJ |
| description | Objective To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology).Design Meta-analysis of RCT-DUPLICATE data.Data sources Trials included in RCT-DUPLICATE, a demonstration project that emulated 32 randomised controlled trials using three real world data sources: Optum Clinformatics Data Mart, 2004-19; IBM MarketScan, 2003-17; and subsets of Medicare parts A, B, and D, 2009-17.Eligibility criteria for selecting studies Trials where the primary analysis resulted in a hazard ratio; 29 RCT-RWE study pairs from RCT-DUPLICATE.Results Differences and variation in effect sizes between the results from randomised controlled trials and real world evidence studies were investigated. Most of the heterogeneity in effect estimates between the RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: treatment started in hospital (which does not appear in health insurance claims data), discontinuation of some baseline treatments at randomisation (which would have been an unusual care decision in clinical practice), and delayed onset of drug effects (which would be under-reported in real world clinical practice because of the relatively short persistence of the treatment). Adding the three emulation differences to the meta-regression reduced heterogeneity from 1.9 to almost 1 (absence of heterogeneity).Conclusions This analysis suggests that a substantial proportion of the observed variation between results from randomised controlled trials and real world evidence studies can be attributed to differences in design emulation. |
| format | Article |
| id | doaj-art-9e3119892f264705bccfe5525e3af4e3 |
| institution | OA Journals |
| issn | 2754-0413 |
| language | English |
| publishDate | 2024-02-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Medicine |
| spelling | doaj-art-9e3119892f264705bccfe5525e3af4e32025-08-20T02:11:26ZengBMJ Publishing GroupBMJ Medicine2754-04132024-02-013110.1136/bmjmed-2023-000709Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE dataSebastian Schneeweiss0Shirley V Wang1Rachel Heyard2Leonhard Held3Division of Pharmacoepidemiology, Brigham and Womems Hospital Harvard Medical School, Boston, Massachusetts, USADivision of Pharmacoepidemiology, Brigham and Womems Hospital Harvard Medical School, Boston, Massachusetts, USACenter for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, SwitzerlandCenter for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, SwitzerlandObjective To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology).Design Meta-analysis of RCT-DUPLICATE data.Data sources Trials included in RCT-DUPLICATE, a demonstration project that emulated 32 randomised controlled trials using three real world data sources: Optum Clinformatics Data Mart, 2004-19; IBM MarketScan, 2003-17; and subsets of Medicare parts A, B, and D, 2009-17.Eligibility criteria for selecting studies Trials where the primary analysis resulted in a hazard ratio; 29 RCT-RWE study pairs from RCT-DUPLICATE.Results Differences and variation in effect sizes between the results from randomised controlled trials and real world evidence studies were investigated. Most of the heterogeneity in effect estimates between the RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: treatment started in hospital (which does not appear in health insurance claims data), discontinuation of some baseline treatments at randomisation (which would have been an unusual care decision in clinical practice), and delayed onset of drug effects (which would be under-reported in real world clinical practice because of the relatively short persistence of the treatment). Adding the three emulation differences to the meta-regression reduced heterogeneity from 1.9 to almost 1 (absence of heterogeneity).Conclusions This analysis suggests that a substantial proportion of the observed variation between results from randomised controlled trials and real world evidence studies can be attributed to differences in design emulation.https://bmjmedicine.bmj.com/content/3/1/e000709.full |
| spellingShingle | Sebastian Schneeweiss Shirley V Wang Rachel Heyard Leonhard Held Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data BMJ Medicine |
| title | Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data |
| title_full | Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data |
| title_fullStr | Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data |
| title_full_unstemmed | Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data |
| title_short | Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data |
| title_sort | design differences and variation in results between randomised trials and non randomised emulations meta analysis of rct duplicate data |
| url | https://bmjmedicine.bmj.com/content/3/1/e000709.full |
| work_keys_str_mv | AT sebastianschneeweiss designdifferencesandvariationinresultsbetweenrandomisedtrialsandnonrandomisedemulationsmetaanalysisofrctduplicatedata AT shirleyvwang designdifferencesandvariationinresultsbetweenrandomisedtrialsandnonrandomisedemulationsmetaanalysisofrctduplicatedata AT rachelheyard designdifferencesandvariationinresultsbetweenrandomisedtrialsandnonrandomisedemulationsmetaanalysisofrctduplicatedata AT leonhardheld designdifferencesandvariationinresultsbetweenrandomisedtrialsandnonrandomisedemulationsmetaanalysisofrctduplicatedata |