The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade®
Abstract Background Autoimmune disorders have primary manifestations such as joint pain and bowel inflammation but can also have secondary manifestations such as non-infectious uveitis (NIU). A regulatory health authority raised concerns after receiving spontaneous reports for NIU following exposure...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-12-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-024-02428-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850103377714741248 |
|---|---|
| author | James Weaver Erica A. Voss Guy Cafri Kathleen Beyrau Michelle Nashleanas Robert Suruki |
| author_facet | James Weaver Erica A. Voss Guy Cafri Kathleen Beyrau Michelle Nashleanas Robert Suruki |
| author_sort | James Weaver |
| collection | DOAJ |
| description | Abstract Background Autoimmune disorders have primary manifestations such as joint pain and bowel inflammation but can also have secondary manifestations such as non-infectious uveitis (NIU). A regulatory health authority raised concerns after receiving spontaneous reports for NIU following exposure to Remicade®, a biologic therapy with multiple indications for which alternative therapies are available. In assessment of this clinical question, we applied validity diagnostics to support observational data causal inferences. Methods We assessed the risk of NIU among patients exposed to Remicade® compared to alternative biologics. Five databases, four study populations, and four analysis methodologies were used to estimate 80 potential treatment effects, with 20 pre-specified as primary. The study populations included inflammatory bowel conditions Crohn’s disease or ulcerative colitis (IBD), ankylosing spondylitis (AS), psoriatic conditions plaque psoriasis or psoriatic arthritis (PsO/PsA), and rheumatoid arthritis (RA). We conducted four analysis strategies intended to address limitations of causal estimation using observational data and applied four diagnostics with pre-specified quantitative rules to evaluate threats to validity from observed and unobserved confounding. We also qualitatively assessed post-propensity score matching representativeness, and bias susceptibility from outcome misclassification. We fit Cox proportional-hazards models, conditioned on propensity score-matched sets, to estimate the on-treatment risk of NIU among Remicade® initiators versus alternatives. Estimates from analyses that passed four validity tests were assessed. Results Of the 80 total analyses and the 20 analyses pre-specified as primary, 24% and 20% passed diagnostics, respectively. Among patients with IBD, we observed no evidence of increased risk for NIU relative to other similarly indicated biologics (pooled hazard ratio [HR] 0.75, 95% confidence interval [CI] 0.38–1.40). For patients with RA, we observed no increased risk relative to similarly indicated biologics, although results were imprecise (HR: 1.23, 95% CI 0.14–10.47). Conclusions We applied validity diagnostics on a heterogenous, observational setting to answer a specific research question. The results indicated that safety effect estimates from many analyses would be inappropriate to interpret as causal, given the data available and methods employed. Validity diagnostics should always be used to determine if the design and analysis are of sufficient quality to support causal inferences. The clinical implications of our findings on IBD suggests that, if an increased risk exists, it is unlikely to be greater than 40% given the 1.40 upper bound of the pooled HR confidence interval. |
| format | Article |
| id | doaj-art-dbcc0da499ae4f059f96d49d490cfccf |
| institution | DOAJ |
| issn | 1471-2288 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj-art-dbcc0da499ae4f059f96d49d490cfccf2025-08-20T02:39:34ZengBMCBMC Medical Research Methodology1471-22882024-12-0124112010.1186/s12874-024-02428-7The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade®James Weaver0Erica A. Voss1Guy Cafri2Kathleen Beyrau3Michelle Nashleanas4Robert Suruki5Janssen Research & Development LLC, Global Epidemiology OrganizationJanssen Research & Development LLC, Global Epidemiology OrganizationJohnson & Johnson MedTech Epidemiology and Real-World Data SciencesJohnson & Johnson Global Medical SafetyJohnson & Johnson Global Medical SafetyJanssen Research & Development LLC, Global Epidemiology OrganizationAbstract Background Autoimmune disorders have primary manifestations such as joint pain and bowel inflammation but can also have secondary manifestations such as non-infectious uveitis (NIU). A regulatory health authority raised concerns after receiving spontaneous reports for NIU following exposure to Remicade®, a biologic therapy with multiple indications for which alternative therapies are available. In assessment of this clinical question, we applied validity diagnostics to support observational data causal inferences. Methods We assessed the risk of NIU among patients exposed to Remicade® compared to alternative biologics. Five databases, four study populations, and four analysis methodologies were used to estimate 80 potential treatment effects, with 20 pre-specified as primary. The study populations included inflammatory bowel conditions Crohn’s disease or ulcerative colitis (IBD), ankylosing spondylitis (AS), psoriatic conditions plaque psoriasis or psoriatic arthritis (PsO/PsA), and rheumatoid arthritis (RA). We conducted four analysis strategies intended to address limitations of causal estimation using observational data and applied four diagnostics with pre-specified quantitative rules to evaluate threats to validity from observed and unobserved confounding. We also qualitatively assessed post-propensity score matching representativeness, and bias susceptibility from outcome misclassification. We fit Cox proportional-hazards models, conditioned on propensity score-matched sets, to estimate the on-treatment risk of NIU among Remicade® initiators versus alternatives. Estimates from analyses that passed four validity tests were assessed. Results Of the 80 total analyses and the 20 analyses pre-specified as primary, 24% and 20% passed diagnostics, respectively. Among patients with IBD, we observed no evidence of increased risk for NIU relative to other similarly indicated biologics (pooled hazard ratio [HR] 0.75, 95% confidence interval [CI] 0.38–1.40). For patients with RA, we observed no increased risk relative to similarly indicated biologics, although results were imprecise (HR: 1.23, 95% CI 0.14–10.47). Conclusions We applied validity diagnostics on a heterogenous, observational setting to answer a specific research question. The results indicated that safety effect estimates from many analyses would be inappropriate to interpret as causal, given the data available and methods employed. Validity diagnostics should always be used to determine if the design and analysis are of sufficient quality to support causal inferences. The clinical implications of our findings on IBD suggests that, if an increased risk exists, it is unlikely to be greater than 40% given the 1.40 upper bound of the pooled HR confidence interval.https://doi.org/10.1186/s12874-024-02428-7Observational studyCausal inferenceAutoimmune disordersBiologic agentsNon-infectious uveitis |
| spellingShingle | James Weaver Erica A. Voss Guy Cafri Kathleen Beyrau Michelle Nashleanas Robert Suruki The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade® BMC Medical Research Methodology Observational study Causal inference Autoimmune disorders Biologic agents Non-infectious uveitis |
| title | The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade® |
| title_full | The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade® |
| title_fullStr | The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade® |
| title_full_unstemmed | The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade® |
| title_short | The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade® |
| title_sort | necessity of validity diagnostics when drawing causal inferences from observational data lessons from a multi database evaluation of the risk of non infectious uveitis among patients exposed to remicade r |
| topic | Observational study Causal inference Autoimmune disorders Biologic agents Non-infectious uveitis |
| url | https://doi.org/10.1186/s12874-024-02428-7 |
| work_keys_str_mv | AT jamesweaver thenecessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT ericaavoss thenecessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT guycafri thenecessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT kathleenbeyrau thenecessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT michellenashleanas thenecessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT robertsuruki thenecessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT jamesweaver necessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT ericaavoss necessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT guycafri necessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT kathleenbeyrau necessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT michellenashleanas necessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade AT robertsuruki necessityofvaliditydiagnosticswhendrawingcausalinferencesfromobservationaldatalessonsfromamultidatabaseevaluationoftheriskofnoninfectiousuveitisamongpatientsexposedtoremicade |