EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS
This study investigates the application of large language models, particularly ChatGPT, in the extraction and structuring of medical information from free-text patient reports. The authors explore two distinct methods: a zero-shot extraction approach and a schema-based extraction approach. The datas...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Polish Association for Knowledge Promotion
2024-12-01
|
| Series: | Applied Computer Science |
| Subjects: | |
| Online Access: | https://ph.pollub.pl/index.php/acs/article/view/6532 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841552800231718912 |
|---|---|
| author | Zakaria KADDARI Ikram El HACHMI Jamal BERRICH Rim AMRANI Toumi BOUCHENTOUF |
| author_facet | Zakaria KADDARI Ikram El HACHMI Jamal BERRICH Rim AMRANI Toumi BOUCHENTOUF |
| author_sort | Zakaria KADDARI |
| collection | DOAJ |
| description | This study investigates the application of large language models, particularly ChatGPT, in the extraction and structuring of medical information from free-text patient reports. The authors explore two distinct methods: a zero-shot extraction approach and a schema-based extraction approach. The dataset, consisting of 1230 anonymized French medical reports from the Department of Neonatology of the Mohammed VI University Hospital, served as the basis for these experiments. The findings indicate that while ChatGPT demonstrates a significant capability in structuring medical data, certain challenges remain, particularly with complex and non-standardized text formats. The authors evaluate the model's performance using precision, recall, and F1 score metrics, providing a comprehensive assessment of its applicability in clinical settings.
|
| format | Article |
| id | doaj-art-c88acde7451244988cfe4d1dd4c08da3 |
| institution | Kabale University |
| issn | 2353-6977 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Polish Association for Knowledge Promotion |
| record_format | Article |
| series | Applied Computer Science |
| spelling | doaj-art-c88acde7451244988cfe4d1dd4c08da32025-01-09T12:44:46ZengPolish Association for Knowledge PromotionApplied Computer Science2353-69772024-12-0120410.35784/acs-2024-44EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODSZakaria KADDARI0https://orcid.org/0000-0003-4034-5612Ikram El HACHMI1https://orcid.org/0009-0008-7928-3088Jamal BERRICH2https://orcid.org/0000-0001-8443-7223Rim AMRANI3https://orcid.org/0000-0003-3906-5533Toumi BOUCHENTOUF4https://orcid.org/0000-0002-2689-8678Université Mohammed Premier, National School of Applied Sciences, LaRSA laboratory, AIRES teamUniversité Mohammed Premier, Faculty of Medicine and Pharmacy OujdaUniversité Mohammed Premier, Faculty of Medicine and Pharmacy OujdaUniversité Mohammed Premier, Faculty of Medicine and Pharmacy OujdaUniversité Mohammed Premier, Faculty of Medicine and Pharmacy OujdaThis study investigates the application of large language models, particularly ChatGPT, in the extraction and structuring of medical information from free-text patient reports. The authors explore two distinct methods: a zero-shot extraction approach and a schema-based extraction approach. The dataset, consisting of 1230 anonymized French medical reports from the Department of Neonatology of the Mohammed VI University Hospital, served as the basis for these experiments. The findings indicate that while ChatGPT demonstrates a significant capability in structuring medical data, certain challenges remain, particularly with complex and non-standardized text formats. The authors evaluate the model's performance using precision, recall, and F1 score metrics, providing a comprehensive assessment of its applicability in clinical settings. https://ph.pollub.pl/index.php/acs/article/view/6532Medical Information Extraction Large Language Models ChatGPT schema-based extraction |
| spellingShingle | Zakaria KADDARI Ikram El HACHMI Jamal BERRICH Rim AMRANI Toumi BOUCHENTOUF EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS Applied Computer Science Medical Information Extraction Large Language Models ChatGPT schema-based extraction |
| title | EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS |
| title_full | EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS |
| title_fullStr | EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS |
| title_full_unstemmed | EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS |
| title_short | EVALUATING LARGE LANGUAGE MODELS FOR MEDICAL INFORMATION EXTRACTION: A COMPARATIVE STUDY OF ZERO-SHOT AND SCHEMA-BASED METHODS |
| title_sort | evaluating large language models for medical information extraction a comparative study of zero shot and schema based methods |
| topic | Medical Information Extraction Large Language Models ChatGPT schema-based extraction |
| url | https://ph.pollub.pl/index.php/acs/article/view/6532 |
| work_keys_str_mv | AT zakariakaddari evaluatinglargelanguagemodelsformedicalinformationextractionacomparativestudyofzeroshotandschemabasedmethods AT ikramelhachmi evaluatinglargelanguagemodelsformedicalinformationextractionacomparativestudyofzeroshotandschemabasedmethods AT jamalberrich evaluatinglargelanguagemodelsformedicalinformationextractionacomparativestudyofzeroshotandschemabasedmethods AT rimamrani evaluatinglargelanguagemodelsformedicalinformationextractionacomparativestudyofzeroshotandschemabasedmethods AT toumibouchentouf evaluatinglargelanguagemodelsformedicalinformationextractionacomparativestudyofzeroshotandschemabasedmethods |