Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare
Background Systematic reviews (SRs) are being published at an accelerated rate. Decision-makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision-makers (eg, practitioners, policymakers, researchers) use SRs to inform deci...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2024-07-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/14/7/e084124.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850067643199913984 |
|---|---|
| author | Beverley Shea Andrea C Tricco Clare Ardern Dawid Pieper Ba Pham Salmaan Kanji Yuan Chi Sera Whitelaw Carole Lunny Areti-Angeliki Veroniki Nicola Ferri Jia He (Janet) Zhang Jasmeen Dourka Emma K Reid Ebrahim Bagheri |
| author_facet | Beverley Shea Andrea C Tricco Clare Ardern Dawid Pieper Ba Pham Salmaan Kanji Yuan Chi Sera Whitelaw Carole Lunny Areti-Angeliki Veroniki Nicola Ferri Jia He (Janet) Zhang Jasmeen Dourka Emma K Reid Ebrahim Bagheri |
| author_sort | Beverley Shea |
| collection | DOAJ |
| description | Background Systematic reviews (SRs) are being published at an accelerated rate. Decision-makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision-makers (eg, practitioners, policymakers, researchers) use SRs to inform decision-making and to explore the potential role of a proposed artificial intelligence (AI) tool to assist in critical appraisal and choosing among SRs.Methods We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks.Results Our survey response rate was lower than expected (7.9% of distributed emails). Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision-making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR among multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%) or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, and recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs.Conclusions Given the identified barriers of using SR evidence, an AI tool to facilitate comparison of the relevance of SRs, the search and methodological quality, could help users efficiently choose among SRs and make healthcare decisions. |
| format | Article |
| id | doaj-art-5c4c750232a3401ca3fc4f73f29f19cd |
| institution | DOAJ |
| issn | 2044-6055 |
| language | English |
| publishDate | 2024-07-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-5c4c750232a3401ca3fc4f73f29f19cd2025-08-20T02:48:15ZengBMJ Publishing GroupBMJ Open2044-60552024-07-0114710.1136/bmjopen-2024-084124Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcareBeverley Shea0Andrea C Tricco1Clare Ardern2Dawid Pieper3Ba Pham4Salmaan Kanji5Yuan Chi6Sera Whitelaw7Carole Lunny8Areti-Angeliki Veroniki9Nicola Ferri10Jia He (Janet) Zhang11Jasmeen Dourka12Emma K Reid13Ebrahim Bagheri1413 University of Ottawa, Ottawa, Ontario, CanadaDalla Lana School of Public Health and Li Ka Shing Knowledge Institute, University of Toronto and St. Michael`s Hospital, Unity Health, Toronto, Ontario, Canada15 Department of Family Practice, The University of British Columbia—Vancouver Campus, Vancouver, British Columbia, CanadaInstitute for Research in Operative Medicine, Witten/Herdecke University, Witten, Germanyresearch coordinator10 Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada5 Yealth Network, Beijing Health Technology Co., Ltd, Beijing, China3 Faculty of Medicine and Health Sciences, McGill University, Montreal, Québec, Canada2 Evidence Synthesis, Precisionheor LLC, Vancouver, British Columbia, Canadascientist6 Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy7 Anesthesiology, Pharmacology & Therapeutics, The University of British Columbia, Vancouver, British Columbia, Canada14 Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael`s Hospital, Toronto, Ontario, Canada4 Department of Pharmacy, Nova Scotia Health Authority, Halifax, Nova Scotia, Canada17 Department of Electrical and Computer Engineering, Toronto Metropolitan University, Toronto, Ontario, CanadaBackground Systematic reviews (SRs) are being published at an accelerated rate. Decision-makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision-makers (eg, practitioners, policymakers, researchers) use SRs to inform decision-making and to explore the potential role of a proposed artificial intelligence (AI) tool to assist in critical appraisal and choosing among SRs.Methods We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks.Results Our survey response rate was lower than expected (7.9% of distributed emails). Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision-making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR among multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%) or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, and recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs.Conclusions Given the identified barriers of using SR evidence, an AI tool to facilitate comparison of the relevance of SRs, the search and methodological quality, could help users efficiently choose among SRs and make healthcare decisions.https://bmjopen.bmj.com/content/14/7/e084124.full |
| spellingShingle | Beverley Shea Andrea C Tricco Clare Ardern Dawid Pieper Ba Pham Salmaan Kanji Yuan Chi Sera Whitelaw Carole Lunny Areti-Angeliki Veroniki Nicola Ferri Jia He (Janet) Zhang Jasmeen Dourka Emma K Reid Ebrahim Bagheri Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare BMJ Open |
| title | Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare |
| title_full | Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare |
| title_fullStr | Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare |
| title_full_unstemmed | Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare |
| title_short | Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare |
| title_sort | exploring decision makers challenges and strategies when selecting multiple systematic reviews insights for ai decision support tools in healthcare |
| url | https://bmjopen.bmj.com/content/14/7/e084124.full |
| work_keys_str_mv | AT beverleyshea exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT andreactricco exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT clareardern exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT dawidpieper exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT bapham exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT salmaankanji exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT yuanchi exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT serawhitelaw exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT carolelunny exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT aretiangelikiveroniki exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT nicolaferri exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT jiahejanetzhang exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT jasmeendourka exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT emmakreid exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare AT ebrahimbagheri exploringdecisionmakerschallengesandstrategieswhenselectingmultiplesystematicreviewsinsightsforaidecisionsupporttoolsinhealthcare |