Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention
Named Entity Recognition (NER) is a fundamental task in natural language processing that aims to identify and categorize named entities within unstructured text. In recent years, with the development of deep learning techniques, pre-trained language models have been widely used in NER tasks. However...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Big Data and Cognitive Computing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-2289/8/12/179 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846105824244531200 |
|---|---|
| author | Chengzhe Yuan Feiyi Tang Chun Shan Weiqiang Shen Ronghua Lin Chengjie Mao Junxian Li |
| author_facet | Chengzhe Yuan Feiyi Tang Chun Shan Weiqiang Shen Ronghua Lin Chengjie Mao Junxian Li |
| author_sort | Chengzhe Yuan |
| collection | DOAJ |
| description | Named Entity Recognition (NER) is a fundamental task in natural language processing that aims to identify and categorize named entities within unstructured text. In recent years, with the development of deep learning techniques, pre-trained language models have been widely used in NER tasks. However, these models still face limitations in terms of their scalability and adaptability, especially when dealing with complex linguistic phenomena such as nested entities and long-range dependencies. To address these challenges, we propose the MacBERT-BiGRU-Self Attention-Global Pointer (MB-GAP) model, which integrates MacBERT for deep semantic understanding, BiGRU for rich contextual information, self-attention for focusing on relevant parts of the input, and a global pointer mechanism for precise entity boundary detection. By optimizing the number of attention heads and global pointer heads, our model achieves an effective balance between complexity and performance. Extensive experiments on benchmark datasets, including ResumeNER, CLUENER2020, and SCHOLAT-School, demonstrate significant improvements over baseline models. |
| format | Article |
| id | doaj-art-09e356e7b53b473186766bc4fd1cabb5 |
| institution | Kabale University |
| issn | 2504-2289 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Big Data and Cognitive Computing |
| spelling | doaj-art-09e356e7b53b473186766bc4fd1cabb52024-12-27T14:10:47ZengMDPI AGBig Data and Cognitive Computing2504-22892024-12-0181217910.3390/bdcc8120179Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-AttentionChengzhe Yuan0Feiyi Tang1Chun Shan2Weiqiang Shen3Ronghua Lin4Chengjie Mao5Junxian Li6School of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou 510665, ChinaSchool of Information Engineering, Guangzhou Panyu Polytechnic, Guangzhou 511483, ChinaSchool of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou 510665, ChinaSchool of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou 510665, ChinaSchool of Computer Science, South China Normal University, Guangzhou 510631, ChinaSchool of Computer Science, South China Normal University, Guangzhou 510631, ChinaSchool of Information Engineering, Guangzhou Panyu Polytechnic, Guangzhou 511483, ChinaNamed Entity Recognition (NER) is a fundamental task in natural language processing that aims to identify and categorize named entities within unstructured text. In recent years, with the development of deep learning techniques, pre-trained language models have been widely used in NER tasks. However, these models still face limitations in terms of their scalability and adaptability, especially when dealing with complex linguistic phenomena such as nested entities and long-range dependencies. To address these challenges, we propose the MacBERT-BiGRU-Self Attention-Global Pointer (MB-GAP) model, which integrates MacBERT for deep semantic understanding, BiGRU for rich contextual information, self-attention for focusing on relevant parts of the input, and a global pointer mechanism for precise entity boundary detection. By optimizing the number of attention heads and global pointer heads, our model achieves an effective balance between complexity and performance. Extensive experiments on benchmark datasets, including ResumeNER, CLUENER2020, and SCHOLAT-School, demonstrate significant improvements over baseline models.https://www.mdpi.com/2504-2289/8/12/179named entity recognitionMacBERTself-attentionglobal pointer |
| spellingShingle | Chengzhe Yuan Feiyi Tang Chun Shan Weiqiang Shen Ronghua Lin Chengjie Mao Junxian Li Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention Big Data and Cognitive Computing named entity recognition MacBERT self-attention global pointer |
| title | Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention |
| title_full | Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention |
| title_fullStr | Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention |
| title_full_unstemmed | Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention |
| title_short | Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention |
| title_sort | exploring named entity recognition via macbert bigru and global pointer with self attention |
| topic | named entity recognition MacBERT self-attention global pointer |
| url | https://www.mdpi.com/2504-2289/8/12/179 |
| work_keys_str_mv | AT chengzheyuan exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention AT feiyitang exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention AT chunshan exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention AT weiqiangshen exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention AT ronghualin exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention AT chengjiemao exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention AT junxianli exploringnamedentityrecognitionviamacbertbigruandglobalpointerwithselfattention |