Evaluating for learning and sustainability (ELS) framework: a realist synthesis
Abstract Background Learning Health Systems (LHS), in which continuous and equitable improvements support optimization of healthcare practices, outcomes, experience, and costs, offer enormous potential for health system transformation. Within the LHS model, evaluation of health innovations assists i...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-05-01
|
| Series: | BMC Health Services Research |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12913-025-12743-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850273572752195584 |
|---|---|
| author | Marissa Bird Maura MacPhee James Shaw Walter P. Wodchis Lianne Jeffs Tujuanna Austin Frances Bruno Balpreet Panesar Élizabeth Côté Boileau Robert J. Reid Carolyn Steele Gray |
| author_facet | Marissa Bird Maura MacPhee James Shaw Walter P. Wodchis Lianne Jeffs Tujuanna Austin Frances Bruno Balpreet Panesar Élizabeth Côté Boileau Robert J. Reid Carolyn Steele Gray |
| author_sort | Marissa Bird |
| collection | DOAJ |
| description | Abstract Background Learning Health Systems (LHS), in which continuous and equitable improvements support optimization of healthcare practices, outcomes, experience, and costs, offer enormous potential for health system transformation. Within the LHS model, evaluation of health innovations assists in question identification, data collection, and targeted action, which facilitates continuous improvement. Evaluation that catalyzes learning may contribute to health innovation implementation, refinement, and sustainability, however, there is little consensus as to why certain evaluations support learning, while others impede it. Methods Embedded in the implementation science literature, we conducted a realist synthesis to understand evaluative contextual factors and underlying mechanisms that best support health system learning and sustainable implementation of innovations. We sought to understand whether evaluations can ‘work’ to support learning and sustainability, in which contexts, for whom, and why. Working with an Expert Committee comprised of leaders in evaluation, innovation, sustainability, and realist methodology, we followed a five-stage process of: 1. Scoping the Review, 2. Building Theories, 3. Identifying the Evidence, 4. Evidence Selection and Appraisal, and 5. Data Extraction and Synthesis. Our Review Team and Expert Committee participated in iterative cycles of results interpretation and feedback. Results Our synthesis includes 60 articles capturing the mechanisms and contextual factors driving learning and sustainability through evaluation. We found that evaluations that support learning and sustainability incorporate favourable organizational preconditions and focus on implementing rapid cyclical feedback loops that contribute to a culture of innovation and evaluation sustainability. Our findings have been organized into 6 Context-Mechanism-Outcome Configurations (CMOCs): 1. Embracing Risk & Failure; 2. Increasing Capacity for Evaluation; 3. Co-Producing Evaluation; 4. Implementing Learning Feedback Loops; 5. Creating Sustainability Culture; and 6. Becoming a Learning Organization. We have also translated findings into a series of Action Strategies for evaluation implementation to support health systems learning and sustainability. Conclusions We identified key contextual factors and underlying mechanisms that make evaluations ‘work’ (or ‘not work’) to support learning and sustainability. Findings support the operationalization of LHS by translating CMOCs into Action Strategies for those tasked with completing evaluations with a view toward health system learning and innovation sustainability. |
| format | Article |
| id | doaj-art-d54ea79d781a4d7abd88bb8f89c3bc86 |
| institution | OA Journals |
| issn | 1472-6963 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Health Services Research |
| spelling | doaj-art-d54ea79d781a4d7abd88bb8f89c3bc862025-08-20T01:51:27ZengBMCBMC Health Services Research1472-69632025-05-0125111610.1186/s12913-025-12743-4Evaluating for learning and sustainability (ELS) framework: a realist synthesisMarissa Bird0Maura MacPhee1James Shaw2Walter P. Wodchis3Lianne Jeffs4Tujuanna Austin5Frances Bruno6Balpreet Panesar7Élizabeth Côté Boileau8Robert J. Reid9Carolyn Steele Gray10Institute for Better Health, Trillium Health PartnersUniversity of British Columbia School of NursingDepartment of Physical Therapy, Temerty Faculty of Medicine, University of TorontoInstitute for Better Health, Trillium Health PartnersInstitute of Health Policy, Management and Evaluation, University of TorontoInstitute of Health Policy, Management and Evaluation, University of TorontoInstitute of Health Policy, Management and Evaluation, University of TorontoInstitute for Better Health, Trillium Health PartnersDepartment of Health Management, Evaluation and Policy, School of Public Health, University of MontrealInstitute for Better Health, Trillium Health PartnersInstitute of Health Policy, Management and Evaluation, University of TorontoAbstract Background Learning Health Systems (LHS), in which continuous and equitable improvements support optimization of healthcare practices, outcomes, experience, and costs, offer enormous potential for health system transformation. Within the LHS model, evaluation of health innovations assists in question identification, data collection, and targeted action, which facilitates continuous improvement. Evaluation that catalyzes learning may contribute to health innovation implementation, refinement, and sustainability, however, there is little consensus as to why certain evaluations support learning, while others impede it. Methods Embedded in the implementation science literature, we conducted a realist synthesis to understand evaluative contextual factors and underlying mechanisms that best support health system learning and sustainable implementation of innovations. We sought to understand whether evaluations can ‘work’ to support learning and sustainability, in which contexts, for whom, and why. Working with an Expert Committee comprised of leaders in evaluation, innovation, sustainability, and realist methodology, we followed a five-stage process of: 1. Scoping the Review, 2. Building Theories, 3. Identifying the Evidence, 4. Evidence Selection and Appraisal, and 5. Data Extraction and Synthesis. Our Review Team and Expert Committee participated in iterative cycles of results interpretation and feedback. Results Our synthesis includes 60 articles capturing the mechanisms and contextual factors driving learning and sustainability through evaluation. We found that evaluations that support learning and sustainability incorporate favourable organizational preconditions and focus on implementing rapid cyclical feedback loops that contribute to a culture of innovation and evaluation sustainability. Our findings have been organized into 6 Context-Mechanism-Outcome Configurations (CMOCs): 1. Embracing Risk & Failure; 2. Increasing Capacity for Evaluation; 3. Co-Producing Evaluation; 4. Implementing Learning Feedback Loops; 5. Creating Sustainability Culture; and 6. Becoming a Learning Organization. We have also translated findings into a series of Action Strategies for evaluation implementation to support health systems learning and sustainability. Conclusions We identified key contextual factors and underlying mechanisms that make evaluations ‘work’ (or ‘not work’) to support learning and sustainability. Findings support the operationalization of LHS by translating CMOCs into Action Strategies for those tasked with completing evaluations with a view toward health system learning and innovation sustainability.https://doi.org/10.1186/s12913-025-12743-4InnovationEvaluationSustainabilityLearningLearning health systems |
| spellingShingle | Marissa Bird Maura MacPhee James Shaw Walter P. Wodchis Lianne Jeffs Tujuanna Austin Frances Bruno Balpreet Panesar Élizabeth Côté Boileau Robert J. Reid Carolyn Steele Gray Evaluating for learning and sustainability (ELS) framework: a realist synthesis BMC Health Services Research Innovation Evaluation Sustainability Learning Learning health systems |
| title | Evaluating for learning and sustainability (ELS) framework: a realist synthesis |
| title_full | Evaluating for learning and sustainability (ELS) framework: a realist synthesis |
| title_fullStr | Evaluating for learning and sustainability (ELS) framework: a realist synthesis |
| title_full_unstemmed | Evaluating for learning and sustainability (ELS) framework: a realist synthesis |
| title_short | Evaluating for learning and sustainability (ELS) framework: a realist synthesis |
| title_sort | evaluating for learning and sustainability els framework a realist synthesis |
| topic | Innovation Evaluation Sustainability Learning Learning health systems |
| url | https://doi.org/10.1186/s12913-025-12743-4 |
| work_keys_str_mv | AT marissabird evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT mauramacphee evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT jamesshaw evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT walterpwodchis evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT liannejeffs evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT tujuannaaustin evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT francesbruno evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT balpreetpanesar evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT elizabethcoteboileau evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT robertjreid evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis AT carolynsteelegray evaluatingforlearningandsustainabilityelsframeworkarealistsynthesis |