Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts
NLP techniques have been widely adopted in the biomedical domain to perform various text-analytics tasks, such as searching biomedical literature and extracting and deriving new knowledge from biomedical data. One type of biomedical data is clinical texts (e.g., clinical cases and medical records),...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9853208/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850066280964423680 |
|---|---|
| author | Ghadeer Althari Mohammad Alsulmi |
| author_facet | Ghadeer Althari Mohammad Alsulmi |
| author_sort | Ghadeer Althari |
| collection | DOAJ |
| description | NLP techniques have been widely adopted in the biomedical domain to perform various text-analytics tasks, such as searching biomedical literature and extracting and deriving new knowledge from biomedical data. One type of biomedical data is clinical texts (e.g., clinical cases and medical records), which typically contain physicians’ notes about a patient’s health, including previous medical history (symptoms, diseases, lab exams, treatments, etc.), as every visit to the hospital leads to the addition of more information to the patient’s record. Another type of biomedical data is biological articles, which typically discuss and explore a certain phenomenon, such as the behavior of biological entities (e.g., genetic relations and interactions among them) and the roles of specific biological processes in causing diseases (e.g., how genetic amplification can cause tumorous diseases). For both types of biomedical data, negation detection is an essential analytics task that can be applied to identify negated contexts in biomedical text (e.g., detecting the presence of a statement establishing that a patient does not have/fit a certain clinical condition or detecting statements that indicate the nonexistence of certain relations among biological entities). This task has been addressed in prior work by considering a variety of approaches such as rule-based systems, conventional machine-learning classifiers, and deep learning approaches. In this work, we propose applying transformer-based learning for negation detection in biomedical texts. We use pre-trained BERT and other similar models (such as ALBERT, XLNet, and ELECTRA) to address two negation-detection subtasks: negation sentence identification and negation scope recognition. We evaluated our approach using the BioScope corpus and relying on measures such as accuracy, precision, recall, F1, and percentage of correct scopes (PCS). Our findings show the potential of transformer-based learning for negation detection, reaching an accuracy of 99% for negation identification and a PCS of 95% for negation scope recognition. |
| format | Article |
| id | doaj-art-a5c10860d25a42c4b34ed7591995aeda |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-a5c10860d25a42c4b34ed7591995aeda2025-08-20T02:48:46ZengIEEEIEEE Access2169-35362022-01-0110838138382510.1109/ACCESS.2022.31977729853208Exploring Transformer-Based Learning for Negation Detection in Biomedical TextsGhadeer Althari0https://orcid.org/0000-0003-4566-2705Mohammad Alsulmi1https://orcid.org/0000-0002-7900-2273Department of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi ArabiaDepartment of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi ArabiaNLP techniques have been widely adopted in the biomedical domain to perform various text-analytics tasks, such as searching biomedical literature and extracting and deriving new knowledge from biomedical data. One type of biomedical data is clinical texts (e.g., clinical cases and medical records), which typically contain physicians’ notes about a patient’s health, including previous medical history (symptoms, diseases, lab exams, treatments, etc.), as every visit to the hospital leads to the addition of more information to the patient’s record. Another type of biomedical data is biological articles, which typically discuss and explore a certain phenomenon, such as the behavior of biological entities (e.g., genetic relations and interactions among them) and the roles of specific biological processes in causing diseases (e.g., how genetic amplification can cause tumorous diseases). For both types of biomedical data, negation detection is an essential analytics task that can be applied to identify negated contexts in biomedical text (e.g., detecting the presence of a statement establishing that a patient does not have/fit a certain clinical condition or detecting statements that indicate the nonexistence of certain relations among biological entities). This task has been addressed in prior work by considering a variety of approaches such as rule-based systems, conventional machine-learning classifiers, and deep learning approaches. In this work, we propose applying transformer-based learning for negation detection in biomedical texts. We use pre-trained BERT and other similar models (such as ALBERT, XLNet, and ELECTRA) to address two negation-detection subtasks: negation sentence identification and negation scope recognition. We evaluated our approach using the BioScope corpus and relying on measures such as accuracy, precision, recall, F1, and percentage of correct scopes (PCS). Our findings show the potential of transformer-based learning for negation detection, reaching an accuracy of 99% for negation identification and a PCS of 95% for negation scope recognition.https://ieeexplore.ieee.org/document/9853208/Health informaticsbiomedical text analyticsmachine learningnatural language processingtext classification |
| spellingShingle | Ghadeer Althari Mohammad Alsulmi Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts IEEE Access Health informatics biomedical text analytics machine learning natural language processing text classification |
| title | Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts |
| title_full | Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts |
| title_fullStr | Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts |
| title_full_unstemmed | Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts |
| title_short | Exploring Transformer-Based Learning for Negation Detection in Biomedical Texts |
| title_sort | exploring transformer based learning for negation detection in biomedical texts |
| topic | Health informatics biomedical text analytics machine learning natural language processing text classification |
| url | https://ieeexplore.ieee.org/document/9853208/ |
| work_keys_str_mv | AT ghadeeralthari exploringtransformerbasedlearningfornegationdetectioninbiomedicaltexts AT mohammadalsulmi exploringtransformerbasedlearningfornegationdetectioninbiomedicaltexts |