PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models
Abstract Clinical trial matching is the task of identifying trials for which patients may be eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. Thi...
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-10-01
|
| Series: | npj Digital Medicine |
| Online Access: | https://doi.org/10.1038/s41746-024-01274-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850179137362198528 |
|---|---|
| author | Shashi Gupta Aditya Basu Mauro Nievas Jerrin Thomas Nathan Wolfrath Adhitya Ramamurthi Bradley Taylor Anai N. Kothari Regina Schwind Therica M. Miller Sorena Nadaf-Rahrov Yanshan Wang Hrituraj Singh |
| author_facet | Shashi Gupta Aditya Basu Mauro Nievas Jerrin Thomas Nathan Wolfrath Adhitya Ramamurthi Bradley Taylor Anai N. Kothari Regina Schwind Therica M. Miller Sorena Nadaf-Rahrov Yanshan Wang Hrituraj Singh |
| author_sort | Shashi Gupta |
| collection | DOAJ |
| description | Abstract Clinical trial matching is the task of identifying trials for which patients may be eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. This process also results in many patients missing out on potential therapeutic options. Recent advancements in Large Language Models (LLMs) have made automating patient-trial matching possible, as shown in multiple concurrent research studies. However, the current approaches are confined to constrained, often synthetic, datasets that do not adequately mirror the complexities encountered in real-world medical data. In this study, we present an end-to-end large-scale empirical evaluation of a clinical trial matching system and validate it using real-world EHRs. We perform comprehensive experiments with proprietary LLMs and our custom fine-tuned model called OncoLLM and show that OncoLLM outperforms GPT-3.5 and matches the performance of qualified medical doctors for clinical trial matching. |
| format | Article |
| id | doaj-art-182275fd80c34579831829800e2db4cd |
| institution | OA Journals |
| issn | 2398-6352 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Digital Medicine |
| spelling | doaj-art-182275fd80c34579831829800e2db4cd2025-08-20T02:18:35ZengNature Portfolionpj Digital Medicine2398-63522024-10-017111210.1038/s41746-024-01274-7PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language modelsShashi Gupta0Aditya Basu1Mauro Nievas2Jerrin Thomas3Nathan Wolfrath4Adhitya Ramamurthi5Bradley Taylor6Anai N. Kothari7Regina Schwind8Therica M. Miller9Sorena Nadaf-Rahrov10Yanshan Wang11Hrituraj Singh12Triomics ResearchTriomics ResearchTriomics ResearchTriomics ResearchMedical College of WisconsinMedical College of WisconsinMedical College of WisconsinMedical College of WisconsinTriomics ResearchIcahn School of Medicine at Mount SinaiCancer Informatics For Cancer CentersUniversity of PittsburghTriomics ResearchAbstract Clinical trial matching is the task of identifying trials for which patients may be eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. This process also results in many patients missing out on potential therapeutic options. Recent advancements in Large Language Models (LLMs) have made automating patient-trial matching possible, as shown in multiple concurrent research studies. However, the current approaches are confined to constrained, often synthetic, datasets that do not adequately mirror the complexities encountered in real-world medical data. In this study, we present an end-to-end large-scale empirical evaluation of a clinical trial matching system and validate it using real-world EHRs. We perform comprehensive experiments with proprietary LLMs and our custom fine-tuned model called OncoLLM and show that OncoLLM outperforms GPT-3.5 and matches the performance of qualified medical doctors for clinical trial matching.https://doi.org/10.1038/s41746-024-01274-7 |
| spellingShingle | Shashi Gupta Aditya Basu Mauro Nievas Jerrin Thomas Nathan Wolfrath Adhitya Ramamurthi Bradley Taylor Anai N. Kothari Regina Schwind Therica M. Miller Sorena Nadaf-Rahrov Yanshan Wang Hrituraj Singh PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models npj Digital Medicine |
| title | PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models |
| title_full | PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models |
| title_fullStr | PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models |
| title_full_unstemmed | PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models |
| title_short | PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models |
| title_sort | prism patient records interpretation for semantic clinical trial matching system using large language models |
| url | https://doi.org/10.1038/s41746-024-01274-7 |
| work_keys_str_mv | AT shashigupta prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT adityabasu prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT mauronievas prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT jerrinthomas prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT nathanwolfrath prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT adhityaramamurthi prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT bradleytaylor prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT anainkothari prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT reginaschwind prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT thericammiller prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT sorenanadafrahrov prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT yanshanwang prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels AT hriturajsingh prismpatientrecordsinterpretationforsemanticclinicaltrialmatchingsystemusinglargelanguagemodels |