Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study
BackgroundReal-world COVID-19 vaccine effectiveness (VE) studies are investigating exposures of increasing complexity accounting for time since vaccination. These studies require methods that adjust for the confounding that arises when morbidities and demographics are associa...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-01-01
|
| Series: | JMIR Formative Research |
| Online Access: | https://formative.jmir.org/2025/1/e58981 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832584078602272768 |
|---|---|
| author | Elizabeth AK Rowley Patrick K Mitchell Duck-Hye Yang Ned Lewis Brian E Dixon Gabriela Vazquez-Benitez William F Fadel Inih J Essien Allison L Naleway Edward Stenehjem Toan C Ong Manjusha Gaglani Karthik Natarajan Peter Embi Ryan E Wiegand Ruth Link-Gelles Mark W Tenforde Bruce Fireman |
| author_facet | Elizabeth AK Rowley Patrick K Mitchell Duck-Hye Yang Ned Lewis Brian E Dixon Gabriela Vazquez-Benitez William F Fadel Inih J Essien Allison L Naleway Edward Stenehjem Toan C Ong Manjusha Gaglani Karthik Natarajan Peter Embi Ryan E Wiegand Ruth Link-Gelles Mark W Tenforde Bruce Fireman |
| author_sort | Elizabeth AK Rowley |
| collection | DOAJ |
| description |
BackgroundReal-world COVID-19 vaccine effectiveness (VE) studies are investigating exposures of increasing complexity accounting for time since vaccination. These studies require methods that adjust for the confounding that arises when morbidities and demographics are associated with vaccination and the risk of outcome events. Methods based on propensity scores (PS) are well-suited to this when the exposure is dichotomous, but present challenges when the exposure is multinomial.
ObjectiveThis simulation study aimed to investigate alternative methods to adjust for confounding in VE studies that have a test-negative design.
MethodsAdjustment for a disease risk score (DRS) is compared with multivariable logistic regression. Both stratification on the DRS and direct covariate adjustment of the DRS are examined. Multivariable logistic regression with all the covariates and with a limited subset of key covariates is considered. The performance of VE estimators is evaluated across a multinomial vaccination exposure in simulated datasets.
ResultsBias in VE estimates from multivariable models ranged from –5.3% to 6.1% across 4 levels of vaccination. Standard errors of VE estimates were unbiased, and 95% coverage probabilities were attained in most scenarios. The lowest coverage in the multivariable scenarios was 93.7% (95% CI 92.2%-95.2%) and occurred in the multivariable model with key covariates, while the highest coverage in the multivariable scenarios was 95.3% (95% CI 94.0%-96.6%) and occurred in the multivariable model with all covariates. Bias in VE estimates from DRS-adjusted models was low, ranging from –2.2% to 4.2%. However, the DRS-adjusted models underestimated the standard errors of VE estimates, with coverage sometimes below the 95% level. The lowest coverage in the DRS scenarios was 87.8% (95% CI 85.8%-89.8%) and occurred in the direct adjustment for the DRS model. The highest coverage in the DRS scenarios was 94.8% (95% CI 93.4%-96.2%) and occurred in the model that stratified on DRS. Although variation in the performance of VE estimates occurred across modeling strategies, variation in performance was also present across exposure groups.
ConclusionsOverall, models using a DRS to adjust for confounding performed adequately but not as well as the multivariable models that adjusted for covariates individually. |
| format | Article |
| id | doaj-art-8ebae8d1856b4de58132a7b92afe85e1 |
| institution | Kabale University |
| issn | 2561-326X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Formative Research |
| spelling | doaj-art-8ebae8d1856b4de58132a7b92afe85e12025-01-27T19:30:54ZengJMIR PublicationsJMIR Formative Research2561-326X2025-01-019e5898110.2196/58981Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation StudyElizabeth AK Rowleyhttps://orcid.org/0000-0002-0593-1096Patrick K Mitchellhttps://orcid.org/0000-0001-6848-0846Duck-Hye Yanghttps://orcid.org/0009-0006-3493-1550Ned Lewishttps://orcid.org/0009-0003-1862-3062Brian E Dixonhttps://orcid.org/0000-0002-1121-0607Gabriela Vazquez-Benitezhttps://orcid.org/0000-0002-6226-3207William F Fadelhttps://orcid.org/0000-0002-0292-6734Inih J Essienhttps://orcid.org/0000-0003-0775-8255Allison L Nalewayhttps://orcid.org/0000-0001-5747-4643Edward Stenehjemhttps://orcid.org/0000-0001-6065-9964Toan C Onghttps://orcid.org/0000-0001-6787-1407Manjusha Gaglanihttps://orcid.org/0000-0002-3952-9230Karthik Natarajanhttps://orcid.org/0000-0002-9066-9431Peter Embihttps://orcid.org/0000-0002-7733-0847Ryan E Wiegandhttps://orcid.org/0000-0002-9486-1850Ruth Link-Gelleshttps://orcid.org/0000-0002-9617-806XMark W Tenfordehttps://orcid.org/0000-0002-8702-8393Bruce Firemanhttps://orcid.org/0000-0003-1652-985X BackgroundReal-world COVID-19 vaccine effectiveness (VE) studies are investigating exposures of increasing complexity accounting for time since vaccination. These studies require methods that adjust for the confounding that arises when morbidities and demographics are associated with vaccination and the risk of outcome events. Methods based on propensity scores (PS) are well-suited to this when the exposure is dichotomous, but present challenges when the exposure is multinomial. ObjectiveThis simulation study aimed to investigate alternative methods to adjust for confounding in VE studies that have a test-negative design. MethodsAdjustment for a disease risk score (DRS) is compared with multivariable logistic regression. Both stratification on the DRS and direct covariate adjustment of the DRS are examined. Multivariable logistic regression with all the covariates and with a limited subset of key covariates is considered. The performance of VE estimators is evaluated across a multinomial vaccination exposure in simulated datasets. ResultsBias in VE estimates from multivariable models ranged from –5.3% to 6.1% across 4 levels of vaccination. Standard errors of VE estimates were unbiased, and 95% coverage probabilities were attained in most scenarios. The lowest coverage in the multivariable scenarios was 93.7% (95% CI 92.2%-95.2%) and occurred in the multivariable model with key covariates, while the highest coverage in the multivariable scenarios was 95.3% (95% CI 94.0%-96.6%) and occurred in the multivariable model with all covariates. Bias in VE estimates from DRS-adjusted models was low, ranging from –2.2% to 4.2%. However, the DRS-adjusted models underestimated the standard errors of VE estimates, with coverage sometimes below the 95% level. The lowest coverage in the DRS scenarios was 87.8% (95% CI 85.8%-89.8%) and occurred in the direct adjustment for the DRS model. The highest coverage in the DRS scenarios was 94.8% (95% CI 93.4%-96.2%) and occurred in the model that stratified on DRS. Although variation in the performance of VE estimates occurred across modeling strategies, variation in performance was also present across exposure groups. ConclusionsOverall, models using a DRS to adjust for confounding performed adequately but not as well as the multivariable models that adjusted for covariates individually.https://formative.jmir.org/2025/1/e58981 |
| spellingShingle | Elizabeth AK Rowley Patrick K Mitchell Duck-Hye Yang Ned Lewis Brian E Dixon Gabriela Vazquez-Benitez William F Fadel Inih J Essien Allison L Naleway Edward Stenehjem Toan C Ong Manjusha Gaglani Karthik Natarajan Peter Embi Ryan E Wiegand Ruth Link-Gelles Mark W Tenforde Bruce Fireman Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study JMIR Formative Research |
| title | Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study |
| title_full | Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study |
| title_fullStr | Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study |
| title_full_unstemmed | Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study |
| title_short | Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study |
| title_sort | methods to adjust for confounding in test negative design covid 19 effectiveness studies simulation study |
| url | https://formative.jmir.org/2025/1/e58981 |
| work_keys_str_mv | AT elizabethakrowley methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT patrickkmitchell methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT duckhyeyang methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT nedlewis methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT brianedixon methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT gabrielavazquezbenitez methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT williamffadel methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT inihjessien methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT allisonlnaleway methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT edwardstenehjem methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT toancong methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT manjushagaglani methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT karthiknatarajan methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT peterembi methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT ryanewiegand methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT ruthlinkgelles methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT markwtenforde methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy AT brucefireman methodstoadjustforconfoundingintestnegativedesigncovid19effectivenessstudiessimulationstudy |