Causality Extraction from Medical Text Using Large Language Models (LLMs)
This study explores the potential of natural language models, including large language models, to extract causal relations from medical texts, specifically from clinical practice guidelines (CPGs). The outcomes of causality extraction from clinical practice guidelines for gestational diabetes are pr...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/16/1/13 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588357519015936 |
|---|---|
| author | Seethalakshmi Gopalakrishnan Luciana Garbayo Wlodek Zadrozny |
| author_facet | Seethalakshmi Gopalakrishnan Luciana Garbayo Wlodek Zadrozny |
| author_sort | Seethalakshmi Gopalakrishnan |
| collection | DOAJ |
| description | This study explores the potential of natural language models, including large language models, to extract causal relations from medical texts, specifically from clinical practice guidelines (CPGs). The outcomes of causality extraction from clinical practice guidelines for gestational diabetes are presented, marking a first in the field. The results are reported on a set of experiments using variants of BERT (BioBERT, DistilBERT, and BERT) and using newer large language models (LLMs), namely, GPT-4 and LLAMA2. Our experiments show that BioBERT performed better than other models, including the large language models, with an average F1-score of 0.72. The GPT-4 and LLAMA2 results show similar performance but less consistency. The code and an annotated corpus of causal statements within the clinical practice guidelines for gestational diabetes are released. Extracting causal structures might help identify LLMs’ hallucinations and possibly prevent some medical errors if LLMs are used in patient settings. Some practical extensions of extracting causal statements from medical text would include providing additional diagnostic support based on less frequent cause–effect relationships, identifying possible inconsistencies in medical guidelines, and evaluating the evidence for recommendations. |
| format | Article |
| id | doaj-art-9025313045cc499cb6c028cac125df96 |
| institution | Kabale University |
| issn | 2078-2489 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj-art-9025313045cc499cb6c028cac125df962025-01-24T13:35:08ZengMDPI AGInformation2078-24892024-12-011611310.3390/info16010013Causality Extraction from Medical Text Using Large Language Models (LLMs)Seethalakshmi Gopalakrishnan0Luciana Garbayo1Wlodek Zadrozny2Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28221, USALoscalzo Lab, National School of Public Health, FIOCRUZ, Harvard Medical School, Brigham and Women’s Hospital, Boston, MA 02215, USADepartment of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28221, USAThis study explores the potential of natural language models, including large language models, to extract causal relations from medical texts, specifically from clinical practice guidelines (CPGs). The outcomes of causality extraction from clinical practice guidelines for gestational diabetes are presented, marking a first in the field. The results are reported on a set of experiments using variants of BERT (BioBERT, DistilBERT, and BERT) and using newer large language models (LLMs), namely, GPT-4 and LLAMA2. Our experiments show that BioBERT performed better than other models, including the large language models, with an average F1-score of 0.72. The GPT-4 and LLAMA2 results show similar performance but less consistency. The code and an annotated corpus of causal statements within the clinical practice guidelines for gestational diabetes are released. Extracting causal structures might help identify LLMs’ hallucinations and possibly prevent some medical errors if LLMs are used in patient settings. Some practical extensions of extracting causal statements from medical text would include providing additional diagnostic support based on less frequent cause–effect relationships, identifying possible inconsistencies in medical guidelines, and evaluating the evidence for recommendations.https://www.mdpi.com/2078-2489/16/1/13causality extractionlarge language models (LLMs)GPT-4LLAMA2medical text |
| spellingShingle | Seethalakshmi Gopalakrishnan Luciana Garbayo Wlodek Zadrozny Causality Extraction from Medical Text Using Large Language Models (LLMs) Information causality extraction large language models (LLMs) GPT-4 LLAMA2 medical text |
| title | Causality Extraction from Medical Text Using Large Language Models (LLMs) |
| title_full | Causality Extraction from Medical Text Using Large Language Models (LLMs) |
| title_fullStr | Causality Extraction from Medical Text Using Large Language Models (LLMs) |
| title_full_unstemmed | Causality Extraction from Medical Text Using Large Language Models (LLMs) |
| title_short | Causality Extraction from Medical Text Using Large Language Models (LLMs) |
| title_sort | causality extraction from medical text using large language models llms |
| topic | causality extraction large language models (LLMs) GPT-4 LLAMA2 medical text |
| url | https://www.mdpi.com/2078-2489/16/1/13 |
| work_keys_str_mv | AT seethalakshmigopalakrishnan causalityextractionfrommedicaltextusinglargelanguagemodelsllms AT lucianagarbayo causalityextractionfrommedicaltextusinglargelanguagemodelsllms AT wlodekzadrozny causalityextractionfrommedicaltextusinglargelanguagemodelsllms |