Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation
In order to better utilize of the semantic information contained in the part of speech of words and the contextual information of unnatural language accompanying the appearance of words, a part of speech weighted multi-modal sentiment analysis model with dynamic adjustment of semantic representation...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Science Press (China Science Publishing & Media Ltd.)
2024-05-01
|
| Series: | Shenzhen Daxue xuebao. Ligong ban |
| Subjects: | |
| Online Access: | https://journal.szu.edu.cn/en/#/digest?ArticleID=2616 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849774131522830336 |
|---|---|
| author | HUA Qiang CHEN Zhuo ZHANG Feng DONG Chunru |
| author_facet | HUA Qiang CHEN Zhuo ZHANG Feng DONG Chunru |
| author_sort | HUA Qiang |
| collection | DOAJ |
| description | In order to better utilize of the semantic information contained in the part of speech of words and the contextual information of unnatural language accompanying the appearance of words, a part of speech weighted multi-modal sentiment analysis model with dynamic adjustment of semantic representation (PM-DS) is proposed. The PM-DS model takes natural language as the main body, and uses bidirectional encoder representation from transformer model, generalized autoregressive pre-training model for language understanding (XLNet) and a robustly optimized BERT pretraining approach (RoBERTa) to embed words into text patterns, respectively. A dynamic semantic adjustment module is created to effectively combine natural language and unnatural language information. The part of speech weighting module is designed to extract the part of speech of words and assigned weights to optimize sentiment discrimination. Comparative experimental results with the current advanced models such as tensor fusion network and low-rank multimodal fusion show that the average absolute errors of PW-DS model on public data sets CMU-MOSI and CMU-MOSEI are 0.607 and 0.510, respectively, and the binary classification accuracies are 89.02% and 86.93%, respectively, which is better than the models in the comparative experiments. The effects of different modules on the model are also analyzed through ablation experiments. The experimental results demonstrate that the proposed model is effective to deal with the problem of multi-modal emotion analysis. |
| format | Article |
| id | doaj-art-4cc8cd3cf17241858e02c31e4b5b9c1f |
| institution | DOAJ |
| issn | 1000-2618 |
| language | English |
| publishDate | 2024-05-01 |
| publisher | Science Press (China Science Publishing & Media Ltd.) |
| record_format | Article |
| series | Shenzhen Daxue xuebao. Ligong ban |
| spelling | doaj-art-4cc8cd3cf17241858e02c31e4b5b9c1f2025-08-20T03:01:50ZengScience Press (China Science Publishing & Media Ltd.)Shenzhen Daxue xuebao. Ligong ban1000-26182024-05-0141328329210.3724/SP.J.1249.2024.032831000-2618（2024）03-0283-10Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representationHUA Qiang0CHEN Zhuo1ZHANG Feng2DONG Chunru3Hebei Key Laboratory of Machine Learning and Computational Intelligence, College of Mathematics and Information Science, Hebei University, Baoding 071002, Hebei Province, P.R.ChinaHebei Key Laboratory of Machine Learning and Computational Intelligence, College of Mathematics and Information Science, Hebei University, Baoding 071002, Hebei Province, P.R.ChinaHebei Key Laboratory of Machine Learning and Computational Intelligence, College of Mathematics and Information Science, Hebei University, Baoding 071002, Hebei Province, P.R.ChinaHebei Key Laboratory of Machine Learning and Computational Intelligence, College of Mathematics and Information Science, Hebei University, Baoding 071002, Hebei Province, P.R.ChinaIn order to better utilize of the semantic information contained in the part of speech of words and the contextual information of unnatural language accompanying the appearance of words, a part of speech weighted multi-modal sentiment analysis model with dynamic adjustment of semantic representation (PM-DS) is proposed. The PM-DS model takes natural language as the main body, and uses bidirectional encoder representation from transformer model, generalized autoregressive pre-training model for language understanding (XLNet) and a robustly optimized BERT pretraining approach (RoBERTa) to embed words into text patterns, respectively. A dynamic semantic adjustment module is created to effectively combine natural language and unnatural language information. The part of speech weighting module is designed to extract the part of speech of words and assigned weights to optimize sentiment discrimination. Comparative experimental results with the current advanced models such as tensor fusion network and low-rank multimodal fusion show that the average absolute errors of PW-DS model on public data sets CMU-MOSI and CMU-MOSEI are 0.607 and 0.510, respectively, and the binary classification accuracies are 89.02% and 86.93%, respectively, which is better than the models in the comparative experiments. The effects of different modules on the model are also analyzed through ablation experiments. The experimental results demonstrate that the proposed model is effective to deal with the problem of multi-modal emotion analysis.https://journal.szu.edu.cn/en/#/digest?ArticleID=2616artificial intelligencemultimodal sentiment analysisdynamical adjustment semanticpart of speech weightingvisualization of multimodal vector positionvisualization of part of speech weights |
| spellingShingle | HUA Qiang CHEN Zhuo ZHANG Feng DONG Chunru Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation Shenzhen Daxue xuebao. Ligong ban artificial intelligence multimodal sentiment analysis dynamical adjustment semantic part of speech weighting visualization of multimodal vector position visualization of part of speech weights |
| title | Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation |
| title_full | Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation |
| title_fullStr | Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation |
| title_full_unstemmed | Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation |
| title_short | Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation |
| title_sort | part of speech weighted multi modal emotion analysis model with dynamic adjustment of semantic representation |
| topic | artificial intelligence multimodal sentiment analysis dynamical adjustment semantic part of speech weighting visualization of multimodal vector position visualization of part of speech weights |
| url | https://journal.szu.edu.cn/en/#/digest?ArticleID=2616 |
| work_keys_str_mv | AT huaqiang partofspeechweightedmultimodalemotionanalysismodelwithdynamicadjustmentofsemanticrepresentation AT chenzhuo partofspeechweightedmultimodalemotionanalysismodelwithdynamicadjustmentofsemanticrepresentation AT zhangfeng partofspeechweightedmultimodalemotionanalysismodelwithdynamicadjustmentofsemanticrepresentation AT dongchunru partofspeechweightedmultimodalemotionanalysismodelwithdynamicadjustmentofsemanticrepresentation |