Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention
Intent recognition in few-shot scenarios is a hot research topic in natural language understanding tasks. Aiming at the problems of insufficient consideration of fine-grained features of the text and insufficient training of features in the process of model fine-tuning, the Triple Channel IntentBERT...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10445147/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850066461400236032 |
|---|---|
| author | Di Wu Yuying Zheng Peng Cheng |
| author_facet | Di Wu Yuying Zheng Peng Cheng |
| author_sort | Di Wu |
| collection | DOAJ |
| description | Intent recognition in few-shot scenarios is a hot research topic in natural language understanding tasks. Aiming at the problems of insufficient consideration of fine-grained features of the text and insufficient training of features in the process of model fine-tuning, the Triple Channel IntentBERT and Orthogonality Constrained Multi-Head Attention Model (TMH-IntentBERT) is proposed. The part-of-speech features, word features and keyword features are combined to extract fine-grained features of data. And the a priori knowledge of the text is fully utilized. Context information is captured through multi-head attention to learn diversified representations. At the same time, the context and score vector regularization terms are added to reduce the position and representation redundancy between heads and enhance the diversity. The experimental results show that on the public dataset, the TMH-IntentBERT model has a minimum increase of 0.63%, 0.73%, 0.79%, and 1.10% in accuracy, precision, F1 value and AUROC compared with CONVBERT, TOD-BERT, WikiHowRoBERTA, IntentBERT and DFT++, respectively. |
| format | Article |
| id | doaj-art-3e286d5cbca5490abf786504dde8a94e |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-3e286d5cbca5490abf786504dde8a94e2025-08-20T02:48:45ZengIEEEIEEE Access2169-35362024-01-0112316853169610.1109/ACCESS.2024.336990210445147Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head AttentionDi Wu0https://orcid.org/0000-0001-7563-9739Yuying Zheng1Peng Cheng2https://orcid.org/0009-0000-1867-9101School of Information and Electronic Engineering, Hebei University of Engineering, Hebei, Handan, ChinaSchool of Information and Electronic Engineering, Hebei University of Engineering, Hebei, Handan, ChinaSchool of Information and Electronic Engineering, Hebei University of Engineering, Hebei, Handan, ChinaIntent recognition in few-shot scenarios is a hot research topic in natural language understanding tasks. Aiming at the problems of insufficient consideration of fine-grained features of the text and insufficient training of features in the process of model fine-tuning, the Triple Channel IntentBERT and Orthogonality Constrained Multi-Head Attention Model (TMH-IntentBERT) is proposed. The part-of-speech features, word features and keyword features are combined to extract fine-grained features of data. And the a priori knowledge of the text is fully utilized. Context information is captured through multi-head attention to learn diversified representations. At the same time, the context and score vector regularization terms are added to reduce the position and representation redundancy between heads and enhance the diversity. The experimental results show that on the public dataset, the TMH-IntentBERT model has a minimum increase of 0.63%, 0.73%, 0.79%, and 1.10% in accuracy, precision, F1 value and AUROC compared with CONVBERT, TOD-BERT, WikiHowRoBERTA, IntentBERT and DFT++, respectively.https://ieeexplore.ieee.org/document/10445147/Intention recognitionfew-shotfeature fusionmulti-head attentionIntentBERT |
| spellingShingle | Di Wu Yuying Zheng Peng Cheng Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention IEEE Access Intention recognition few-shot feature fusion multi-head attention IntentBERT |
| title | Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention |
| title_full | Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention |
| title_fullStr | Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention |
| title_full_unstemmed | Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention |
| title_short | Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention |
| title_sort | triple channel feature fusion few shot intent recognition with orthogonality constrained multi head attention |
| topic | Intention recognition few-shot feature fusion multi-head attention IntentBERT |
| url | https://ieeexplore.ieee.org/document/10445147/ |
| work_keys_str_mv | AT diwu triplechannelfeaturefusionfewshotintentrecognitionwithorthogonalityconstrainedmultiheadattention AT yuyingzheng triplechannelfeaturefusionfewshotintentrecognitionwithorthogonalityconstrainedmultiheadattention AT pengcheng triplechannelfeaturefusionfewshotintentrecognitionwithorthogonalityconstrainedmultiheadattention |