Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion
Abstract Background The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-01-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-024-02440-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832594656534200320 |
|---|---|
| author | Michelle Pfaffenlehner Max Behrens Daniela Zöller Kathrin Ungethüm Kai Günther Viktoria Rücker Jens-Peter Reese Peter Heuschmann Miriam Kesselmeier Flavia Remo André Scherag Harald Binder Nadine Binder for the EVA4MII project |
| author_facet | Michelle Pfaffenlehner Max Behrens Daniela Zöller Kathrin Ungethüm Kai Günther Viktoria Rücker Jens-Peter Reese Peter Heuschmann Miriam Kesselmeier Flavia Remo André Scherag Harald Binder Nadine Binder for the EVA4MII project |
| author_sort | Michelle Pfaffenlehner |
| collection | DOAJ |
| description | Abstract Background The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like confounding and bias, similar to prospectively documented observational studies. This study focuses on additional limitations frequently reported in the literature, providing an overview of the challenges and biases inherent to analyzing routine clinical care data, including health claims data (hereafter: routine data). Methods We conducted a literature search on routine data studies in four high-impact journals based on the Journal Citation Reports (JCR) category “Medicine, General & Internal” as of 2022 and three oncology journals, covering articles published from January 2018 to October 2023. Articles were screened and categorized into three scenarios based on their potential to provide meaningful RWE: (1) Burden of Disease, (2) Safety and Risk Group Analysis, and (3) Treatment Comparison. Limitations of this type of data cited in the discussion sections were extracted and classified according to different bias types: main bias categories in non-randomized studies (information bias, reporting bias, selection bias, confounding) and additional routine data-specific challenges (i.e., operationalization, coding, follow-up, missing data, validation, and data quality). These classifications were then ranked by relevance in a focus group meeting of methodological experts. The search was pre-specified and registered in PROSPERO (CRD42023477616). Results In October 2023, 227 articles were identified, 69 were assessed for eligibility, and 39 were included in the review: 11 on the burden of disease, 17 on safety and risk group analysis, and 11 on treatment comparison. Besides typical biases in observational studies, we identified additional challenges specific to RWE frequently mentioned in the discussion sections. The focus group had varied opinions on the limitations of Safety and Risk Group Analysis and Treatment Comparison but agreed on the essential limitations for the Burden of Disease category. Conclusion This review provides a comprehensive overview of potential limitations and biases in analyzing routine data reported in recent high-impact journals. We highlighted key challenges that have high potential to impact analysis results, emphasizing the need for thorough consideration and discussion for meaningful inferences. |
| format | Article |
| id | doaj-art-dd4be428aba14135aafaed215ef4c33a |
| institution | Kabale University |
| issn | 1471-2288 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj-art-dd4be428aba14135aafaed215ef4c33a2025-01-19T12:28:14ZengBMCBMC Medical Research Methodology1471-22882025-01-0125111410.1186/s12874-024-02440-xMethodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussionMichelle Pfaffenlehner0Max Behrens1Daniela Zöller2Kathrin Ungethüm3Kai Günther4Viktoria Rücker5Jens-Peter Reese6Peter Heuschmann7Miriam Kesselmeier8Flavia Remo9André Scherag10Harald Binder11Nadine Binder12for the EVA4MII projectInstitute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of FreiburgInstitute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of FreiburgInstitute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of FreiburgInstitute for Medical Data Sciences, University Hospital WürzburgInstitute for Medical Data Sciences, University Hospital WürzburgInstitute for Clinical Epidemiology and Biometry, University WürzburgInstitute for Medical Data Sciences, University Hospital WürzburgInstitute for Medical Data Sciences, University Hospital WürzburgInstitute of Medical Statistics, Computer and Data Sciences, Friedrich Schiller University & Jena University HospitalInstitute of Medical Statistics, Computer and Data Sciences, Friedrich Schiller University & Jena University HospitalInstitute of Medical Statistics, Computer and Data Sciences, Friedrich Schiller University & Jena University HospitalInstitute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of FreiburgFreiburg Center for Data Analysis, Modeling and AI, University of FreiburgAbstract Background The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like confounding and bias, similar to prospectively documented observational studies. This study focuses on additional limitations frequently reported in the literature, providing an overview of the challenges and biases inherent to analyzing routine clinical care data, including health claims data (hereafter: routine data). Methods We conducted a literature search on routine data studies in four high-impact journals based on the Journal Citation Reports (JCR) category “Medicine, General & Internal” as of 2022 and three oncology journals, covering articles published from January 2018 to October 2023. Articles were screened and categorized into three scenarios based on their potential to provide meaningful RWE: (1) Burden of Disease, (2) Safety and Risk Group Analysis, and (3) Treatment Comparison. Limitations of this type of data cited in the discussion sections were extracted and classified according to different bias types: main bias categories in non-randomized studies (information bias, reporting bias, selection bias, confounding) and additional routine data-specific challenges (i.e., operationalization, coding, follow-up, missing data, validation, and data quality). These classifications were then ranked by relevance in a focus group meeting of methodological experts. The search was pre-specified and registered in PROSPERO (CRD42023477616). Results In October 2023, 227 articles were identified, 69 were assessed for eligibility, and 39 were included in the review: 11 on the burden of disease, 17 on safety and risk group analysis, and 11 on treatment comparison. Besides typical biases in observational studies, we identified additional challenges specific to RWE frequently mentioned in the discussion sections. The focus group had varied opinions on the limitations of Safety and Risk Group Analysis and Treatment Comparison but agreed on the essential limitations for the Burden of Disease category. Conclusion This review provides a comprehensive overview of potential limitations and biases in analyzing routine data reported in recent high-impact journals. We highlighted key challenges that have high potential to impact analysis results, emphasizing the need for thorough consideration and discussion for meaningful inferences.https://doi.org/10.1186/s12874-024-02440-xRapid reviewLimitationBiasRoutine clinical care dataReal-world evidenceEHR |
| spellingShingle | Michelle Pfaffenlehner Max Behrens Daniela Zöller Kathrin Ungethüm Kai Günther Viktoria Rücker Jens-Peter Reese Peter Heuschmann Miriam Kesselmeier Flavia Remo André Scherag Harald Binder Nadine Binder for the EVA4MII project Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion BMC Medical Research Methodology Rapid review Limitation Bias Routine clinical care data Real-world evidence EHR |
| title | Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion |
| title_full | Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion |
| title_fullStr | Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion |
| title_full_unstemmed | Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion |
| title_short | Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion |
| title_sort | methodological challenges using routine clinical care data for real world evidence a rapid review utilizing a systematic literature search and focus group discussion |
| topic | Rapid review Limitation Bias Routine clinical care data Real-world evidence EHR |
| url | https://doi.org/10.1186/s12874-024-02440-x |
| work_keys_str_mv | AT michellepfaffenlehner methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT maxbehrens methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT danielazoller methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT kathrinungethum methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT kaigunther methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT viktoriarucker methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT jenspeterreese methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT peterheuschmann methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT miriamkesselmeier methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT flaviaremo methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT andrescherag methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT haraldbinder methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT nadinebinder methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion AT fortheeva4miiproject methodologicalchallengesusingroutineclinicalcaredataforrealworldevidencearapidreviewutilizingasystematicliteraturesearchandfocusgroupdiscussion |