Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions
Abstract During public health crises such as pandemics, governments must rapidly adopt and implement wide-reaching policies and programs (“public policy interventions”). A key takeaway from the coronavirus disease 2019 (COVID-19) pandemic was that although numerous randomized controlled trials (RCTs...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-01-01
|
| Series: | Health Research Policy and Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12961-024-01271-y |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832594348537020416 |
|---|---|
| author | Unni Gopinathan Ingeborg Elgersma Therese Dalsbø Mona Bjørbæk Atle Fretheim |
| author_facet | Unni Gopinathan Ingeborg Elgersma Therese Dalsbø Mona Bjørbæk Atle Fretheim |
| author_sort | Unni Gopinathan |
| collection | DOAJ |
| description | Abstract During public health crises such as pandemics, governments must rapidly adopt and implement wide-reaching policies and programs (“public policy interventions”). A key takeaway from the coronavirus disease 2019 (COVID-19) pandemic was that although numerous randomized controlled trials (RCTs) focussed on drugs and vaccines, few policy experiments were conducted to evaluate effects of public policy interventions across various sectors on viral transmission and other consequences. Moreover, many quasi-experimental studies were of spurious quality, thus proving unhelpful for informing public policy. The pandemic highlighted the need to continuously develop competence, capacity and a robust legal–ethical foundation for impact evaluations well before crises occur. It raised a crucial question: how can governments in non-crisis times develop capabilities to generate evidence on the effects of public policy interventions, thus enabling a rapid and effective research response during public health crises? We conducted a mapping to explore how government agencies in Norway use RCTs and quasi-experimental methods to strengthen the evidence base for public policy interventions and to identify barriers and facilitators to their use. Contributing to the study were 10 government agencies across sectors such as development assistance, education, health, social welfare, statistics and taxation. Many of these agencies have conducted or commissioned RCTs or quasi-experimental studies in the past 5 years, with evaluations ranging from 1 or 2 to more than 15 per agency. The measures evaluated included organizational, educational and financial interventions and interventions for oversight and sanctions. Some agencies have internal capabilities for designing and conducting evaluations, while others commissioned such studies to universities and other research institutions. Agencies reported examples where enhanced communication among implementers, researchers, ministries and political leaders facilitated impact evaluations, and these lessons offer opportunities for cross-sector knowledge-sharing to help strengthen rigorous evaluations of public policy interventions. Despite their potential, various government agencies report that randomized and quasi-experimental studies face legal, ethical, political and practical barriers that affect their use. For instance, the urgency of politicians to implement policies at scale has led to the discontinuation of trials and hindered learning from their effects. The surveyed agencies stressed the importance of legislation providing clear guidelines on when differential treatment can be justified and when informed consent requirements can be waived, as well as faster and clearer processes for managing privacy concerns related to data access. Crucially, greater political acceptance for systematically and gradually implementing reforms, including using randomization, could strengthen evidence-informed public policy, enhancing the scaling-up of effective interventions and deprioritizing ineffective ones. |
| format | Article |
| id | doaj-art-819b2a4b132540d6a5641f7fe581dff9 |
| institution | Kabale University |
| issn | 1478-4505 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMC |
| record_format | Article |
| series | Health Research Policy and Systems |
| spelling | doaj-art-819b2a4b132540d6a5641f7fe581dff92025-01-19T12:43:36ZengBMCHealth Research Policy and Systems1478-45052025-01-0123111410.1186/s12961-024-01271-yStrengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventionsUnni Gopinathan0Ingeborg Elgersma1Therese Dalsbø2Mona Bjørbæk3Atle Fretheim4Centre for Epidemic Interventions Research, Norwegian Institute of Public HealthCentre for Epidemic Interventions Research, Norwegian Institute of Public HealthNational Institute of Occupational HealthCentre for Epidemic Interventions Research, Norwegian Institute of Public HealthCentre for Epidemic Interventions Research, Norwegian Institute of Public HealthAbstract During public health crises such as pandemics, governments must rapidly adopt and implement wide-reaching policies and programs (“public policy interventions”). A key takeaway from the coronavirus disease 2019 (COVID-19) pandemic was that although numerous randomized controlled trials (RCTs) focussed on drugs and vaccines, few policy experiments were conducted to evaluate effects of public policy interventions across various sectors on viral transmission and other consequences. Moreover, many quasi-experimental studies were of spurious quality, thus proving unhelpful for informing public policy. The pandemic highlighted the need to continuously develop competence, capacity and a robust legal–ethical foundation for impact evaluations well before crises occur. It raised a crucial question: how can governments in non-crisis times develop capabilities to generate evidence on the effects of public policy interventions, thus enabling a rapid and effective research response during public health crises? We conducted a mapping to explore how government agencies in Norway use RCTs and quasi-experimental methods to strengthen the evidence base for public policy interventions and to identify barriers and facilitators to their use. Contributing to the study were 10 government agencies across sectors such as development assistance, education, health, social welfare, statistics and taxation. Many of these agencies have conducted or commissioned RCTs or quasi-experimental studies in the past 5 years, with evaluations ranging from 1 or 2 to more than 15 per agency. The measures evaluated included organizational, educational and financial interventions and interventions for oversight and sanctions. Some agencies have internal capabilities for designing and conducting evaluations, while others commissioned such studies to universities and other research institutions. Agencies reported examples where enhanced communication among implementers, researchers, ministries and political leaders facilitated impact evaluations, and these lessons offer opportunities for cross-sector knowledge-sharing to help strengthen rigorous evaluations of public policy interventions. Despite their potential, various government agencies report that randomized and quasi-experimental studies face legal, ethical, political and practical barriers that affect their use. For instance, the urgency of politicians to implement policies at scale has led to the discontinuation of trials and hindered learning from their effects. The surveyed agencies stressed the importance of legislation providing clear guidelines on when differential treatment can be justified and when informed consent requirements can be waived, as well as faster and clearer processes for managing privacy concerns related to data access. Crucially, greater political acceptance for systematically and gradually implementing reforms, including using randomization, could strengthen evidence-informed public policy, enhancing the scaling-up of effective interventions and deprioritizing ineffective ones.https://doi.org/10.1186/s12961-024-01271-yRCTsPolicy experimentsQuasi-experimentsEvidence-informed policymakingResearch ethicsPolitics |
| spellingShingle | Unni Gopinathan Ingeborg Elgersma Therese Dalsbø Mona Bjørbæk Atle Fretheim Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions Health Research Policy and Systems RCTs Policy experiments Quasi-experiments Evidence-informed policymaking Research ethics Politics |
| title | Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions |
| title_full | Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions |
| title_fullStr | Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions |
| title_full_unstemmed | Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions |
| title_short | Strengthening research preparedness for crises: lessons from Norwegian government agencies in using randomized trials and quasi-experimental methods to evaluate public policy interventions |
| title_sort | strengthening research preparedness for crises lessons from norwegian government agencies in using randomized trials and quasi experimental methods to evaluate public policy interventions |
| topic | RCTs Policy experiments Quasi-experiments Evidence-informed policymaking Research ethics Politics |
| url | https://doi.org/10.1186/s12961-024-01271-y |
| work_keys_str_mv | AT unnigopinathan strengtheningresearchpreparednessforcriseslessonsfromnorwegiangovernmentagenciesinusingrandomizedtrialsandquasiexperimentalmethodstoevaluatepublicpolicyinterventions AT ingeborgelgersma strengtheningresearchpreparednessforcriseslessonsfromnorwegiangovernmentagenciesinusingrandomizedtrialsandquasiexperimentalmethodstoevaluatepublicpolicyinterventions AT theresedalsbø strengtheningresearchpreparednessforcriseslessonsfromnorwegiangovernmentagenciesinusingrandomizedtrialsandquasiexperimentalmethodstoevaluatepublicpolicyinterventions AT monabjørbæk strengtheningresearchpreparednessforcriseslessonsfromnorwegiangovernmentagenciesinusingrandomizedtrialsandquasiexperimentalmethodstoevaluatepublicpolicyinterventions AT atlefretheim strengtheningresearchpreparednessforcriseslessonsfromnorwegiangovernmentagenciesinusingrandomizedtrialsandquasiexperimentalmethodstoevaluatepublicpolicyinterventions |