Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions
Objective: To develop and validate a named entity recognition (NER) model based on BERT-based model trained on Spanish-language corpor, for extracting allergy-related information from unstructured electronic health records. Patients and Methods: The model was fine-tuned using 16,176 manually annotat...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-09-01
|
| Series: | Mayo Clinic Proceedings: Digital Health |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2949761225000513 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849320288597049344 |
|---|---|
| author | Juan Pablo Botero-Aguirre, MS Michael Andrés García-Rivera, MS |
| author_facet | Juan Pablo Botero-Aguirre, MS Michael Andrés García-Rivera, MS |
| author_sort | Juan Pablo Botero-Aguirre, MS |
| collection | DOAJ |
| description | Objective: To develop and validate a named entity recognition (NER) model based on BERT-based model trained on Spanish-language corpor, for extracting allergy-related information from unstructured electronic health records. Patients and Methods: The model was fine-tuned using 16,176 manually annotated allergy-related entities from anonimized patient records (hospitalized patients between January 1, 2021, and June 30, 2024). The data set was divided into training (80%) and testing (20%) subsets, and model performance was evaluated using accuracy, recall, and F1 score. The validated model was applied to another data set with 80,917 medication prescriptions from 5859 hospitalized patients with at least one prescribed medication (during August and September 2024) to detect potential prescription errors. Sensitivity, specificity, and Cohen κ were calculated using manual expert review as the gold standard. Results: The model achieved an accuracy of 87.28% and an F1 score of 0.80. It effectively identified medication names (F1=0.91) and adverse reactions (F1=0.85) but struggled with recommendation-related entities (F1=0.29). The model detected prescription errors in 0.96% of cases, with a sensitivity of 75.73% and specificity of 99.98%. The weighted κ score (0.7797) indicated substantial agreement with expert annotations. Conclusion: The BERT-based model trained on Spanish-language corpora–based NER model demonstrated strong performance in identifying nonallergic cases (specificity, 99.98%; negative predictive value, 99.97%) and showed promise for clinical decision support. Despite moderate sensitivity (75.73%), these results highlight the feasibility of using Spanish-language NER models to enhance medication safety. |
| format | Article |
| id | doaj-art-da11bd4558e94516ada05e6417b74b1d |
| institution | Kabale University |
| issn | 2949-7612 |
| language | English |
| publishDate | 2025-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Mayo Clinic Proceedings: Digital Health |
| spelling | doaj-art-da11bd4558e94516ada05e6417b74b1d2025-08-20T03:50:07ZengElsevierMayo Clinic Proceedings: Digital Health2949-76122025-09-013310024410.1016/j.mcpdig.2025.100244Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication PrescriptionsJuan Pablo Botero-Aguirre, MS0Michael Andrés García-Rivera, MS1Department of Artificial Intelligence, Hospital Pablo Tobón Uribe, Colombia; Correspondence: Address to Juan Pablo Botero-Aguirre, MS, Department of Artificial Intelligence, Hospital Pablo Tobón Uribe, Antioquia 050034, Colombia.Department of Pharmaceutical Services, Hospital Pablo Tobón Uribe, ColombiaObjective: To develop and validate a named entity recognition (NER) model based on BERT-based model trained on Spanish-language corpor, for extracting allergy-related information from unstructured electronic health records. Patients and Methods: The model was fine-tuned using 16,176 manually annotated allergy-related entities from anonimized patient records (hospitalized patients between January 1, 2021, and June 30, 2024). The data set was divided into training (80%) and testing (20%) subsets, and model performance was evaluated using accuracy, recall, and F1 score. The validated model was applied to another data set with 80,917 medication prescriptions from 5859 hospitalized patients with at least one prescribed medication (during August and September 2024) to detect potential prescription errors. Sensitivity, specificity, and Cohen κ were calculated using manual expert review as the gold standard. Results: The model achieved an accuracy of 87.28% and an F1 score of 0.80. It effectively identified medication names (F1=0.91) and adverse reactions (F1=0.85) but struggled with recommendation-related entities (F1=0.29). The model detected prescription errors in 0.96% of cases, with a sensitivity of 75.73% and specificity of 99.98%. The weighted κ score (0.7797) indicated substantial agreement with expert annotations. Conclusion: The BERT-based model trained on Spanish-language corpora–based NER model demonstrated strong performance in identifying nonallergic cases (specificity, 99.98%; negative predictive value, 99.97%) and showed promise for clinical decision support. Despite moderate sensitivity (75.73%), these results highlight the feasibility of using Spanish-language NER models to enhance medication safety.http://www.sciencedirect.com/science/article/pii/S2949761225000513 |
| spellingShingle | Juan Pablo Botero-Aguirre, MS Michael Andrés García-Rivera, MS Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions Mayo Clinic Proceedings: Digital Health |
| title | Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions |
| title_full | Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions |
| title_fullStr | Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions |
| title_full_unstemmed | Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions |
| title_short | Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication Prescriptions |
| title_sort | natural language processing for enhanced clinical decision support in allergy verification for medication prescriptions |
| url | http://www.sciencedirect.com/science/article/pii/S2949761225000513 |
| work_keys_str_mv | AT juanpabloboteroaguirrems naturallanguageprocessingforenhancedclinicaldecisionsupportinallergyverificationformedicationprescriptions AT michaelandresgarciariverams naturallanguageprocessingforenhancedclinicaldecisionsupportinallergyverificationformedicationprescriptions |