A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms
Due to the complexity and technicality of named entity recognition (NER) in the power grid field, existing methods are ineffective at identifying specialized terms in power grid operation record texts. Therefore, this paper proposes a Chinese power violation description entity recognition model base...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/18/2/401 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588560273768448 |
|---|---|
| author | Lingwen Meng Yulin Wang Yuanjun Huang Dingli Ma Xinshan Zhu Shumei Zhang |
| author_facet | Lingwen Meng Yulin Wang Yuanjun Huang Dingli Ma Xinshan Zhu Shumei Zhang |
| author_sort | Lingwen Meng |
| collection | DOAJ |
| description | Due to the complexity and technicality of named entity recognition (NER) in the power grid field, existing methods are ineffective at identifying specialized terms in power grid operation record texts. Therefore, this paper proposes a Chinese power violation description entity recognition model based on word-character fusion and multi-head attention mechanisms. The model first utilizes a collected power grid domain corpus to train a Word2Vec model, which produces static word vector representations. These static word vectors are then integrated with the dynamic character vector features of the input text generated by the BERT model, thereby mitigating the impact of segmentation errors on the NER model and enhancing the model’s ability to identify entity boundaries. The combined vectors are subsequently input into a BiGRU model for learning contextual features. The output from the BiGRU layer is then passed to an attention mechanism layer to obtain enhanced semantic features, which highlight key semantics and improve the model’s contextual understanding ability. Finally, the CRF layer decodes the output to generate the globally optimal label sequence with the highest probability. Experimental results on the constructed power grid field operation violation description dataset demonstrate that the proposed NER model outperforms the traditional BERT-BiLSTM-CRF model, with an average improvement of 1.58% in precision, recall, and F1-score. This demonstrates the effectiveness of the model design and further enhances the accuracy of entity recognition in the power grid domain. |
| format | Article |
| id | doaj-art-7adb8aa40259411591bd3de887ec8268 |
| institution | Kabale University |
| issn | 1996-1073 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj-art-7adb8aa40259411591bd3de887ec82682025-01-24T13:31:21ZengMDPI AGEnergies1996-10732025-01-0118240110.3390/en18020401A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention MechanismsLingwen Meng0Yulin Wang1Yuanjun Huang2Dingli Ma3Xinshan Zhu4Shumei Zhang5Electric Power Research Institute of Guizhou Power Grid Co., Ltd., Guiyang 550002, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSouthern Power Grid Digital Grid Group Co., Ltd., Guizhou Branch, Guiyang 550002, ChinaSouthern Power Grid Digital Grid Group Co., Ltd., Guizhou Branch, Guiyang 550002, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaDue to the complexity and technicality of named entity recognition (NER) in the power grid field, existing methods are ineffective at identifying specialized terms in power grid operation record texts. Therefore, this paper proposes a Chinese power violation description entity recognition model based on word-character fusion and multi-head attention mechanisms. The model first utilizes a collected power grid domain corpus to train a Word2Vec model, which produces static word vector representations. These static word vectors are then integrated with the dynamic character vector features of the input text generated by the BERT model, thereby mitigating the impact of segmentation errors on the NER model and enhancing the model’s ability to identify entity boundaries. The combined vectors are subsequently input into a BiGRU model for learning contextual features. The output from the BiGRU layer is then passed to an attention mechanism layer to obtain enhanced semantic features, which highlight key semantics and improve the model’s contextual understanding ability. Finally, the CRF layer decodes the output to generate the globally optimal label sequence with the highest probability. Experimental results on the constructed power grid field operation violation description dataset demonstrate that the proposed NER model outperforms the traditional BERT-BiLSTM-CRF model, with an average improvement of 1.58% in precision, recall, and F1-score. This demonstrates the effectiveness of the model design and further enhances the accuracy of entity recognition in the power grid domain.https://www.mdpi.com/1996-1073/18/2/401named entity recognition (NER)attention mechanismelectric grid on-site operation violation descriptionword-character fusion |
| spellingShingle | Lingwen Meng Yulin Wang Yuanjun Huang Dingli Ma Xinshan Zhu Shumei Zhang A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms Energies named entity recognition (NER) attention mechanism electric grid on-site operation violation description word-character fusion |
| title | A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms |
| title_full | A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms |
| title_fullStr | A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms |
| title_full_unstemmed | A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms |
| title_short | A Named Entity Recognition Model for Chinese Electricity Violation Descriptions Based on Word-Character Fusion and Multi-Head Attention Mechanisms |
| title_sort | named entity recognition model for chinese electricity violation descriptions based on word character fusion and multi head attention mechanisms |
| topic | named entity recognition (NER) attention mechanism electric grid on-site operation violation description word-character fusion |
| url | https://www.mdpi.com/1996-1073/18/2/401 |
| work_keys_str_mv | AT lingwenmeng anamedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT yulinwang anamedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT yuanjunhuang anamedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT dinglima anamedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT xinshanzhu anamedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT shumeizhang anamedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT lingwenmeng namedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT yulinwang namedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT yuanjunhuang namedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT dinglima namedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT xinshanzhu namedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms AT shumeizhang namedentityrecognitionmodelforchineseelectricityviolationdescriptionsbasedonwordcharacterfusionandmultiheadattentionmechanisms |