RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis
To address the limitation that the Robustly Optimized BERT Pretraining Approach (RoBERTa) may not effectively capture local dependencies and salient features within the text, we propose a feature fusion framework based on RoBERTa’s multi-output architecture. By feeding different outputs o...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11031473/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849422106261979136 |
|---|---|
| author | LiHua Yang Jun Wang WangRen Qiu |
| author_facet | LiHua Yang Jun Wang WangRen Qiu |
| author_sort | LiHua Yang |
| collection | DOAJ |
| description | To address the limitation that the Robustly Optimized BERT Pretraining Approach (RoBERTa) may not effectively capture local dependencies and salient features within the text, we propose a feature fusion framework based on RoBERTa’s multi-output architecture. By feeding different outputs of RoBERTa into Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) networks, the model effectively captures both local static patterns and global contextual dependencies, thereby enhancing its capability to handle complex textual inputs. Additionally, an attention mechanism highlights semantically important features, further improving feature representation and classification performance. To evaluate the effectiveness of the proposed model, we conducted comparative experiments on the Amazon, IMDB, SST-2, and Ceramic datasets. The F1 scores achieved were 94.25%, 92.52%, 91.51%, and 81.32%, respectively, substantially outperforming existing models, particularly in handling domain-specific and nuanced texts. These results demonstrate that the proposed multi-feature fusion model significantly enhances sentiment classification performance. |
| format | Article |
| id | doaj-art-9a624c77fa46442185ccd58b7ea71afc |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-9a624c77fa46442185ccd58b7ea71afc2025-08-20T03:31:14ZengIEEEIEEE Access2169-35362025-01-011310366810368110.1109/ACCESS.2025.357902211031473RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review AnalysisLiHua Yang0https://orcid.org/0000-0002-8296-2769Jun Wang1https://orcid.org/0009-0004-9886-9368WangRen Qiu2https://orcid.org/0000-0001-7659-8553School of Information Engineering, Jingdezhen Ceramic University, Jingdezhen, Jiangxi, ChinaSchool of Information Engineering, Jingdezhen Ceramic University, Jingdezhen, Jiangxi, ChinaSchool of Information Engineering, Jingdezhen Ceramic University, Jingdezhen, Jiangxi, ChinaTo address the limitation that the Robustly Optimized BERT Pretraining Approach (RoBERTa) may not effectively capture local dependencies and salient features within the text, we propose a feature fusion framework based on RoBERTa’s multi-output architecture. By feeding different outputs of RoBERTa into Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) networks, the model effectively captures both local static patterns and global contextual dependencies, thereby enhancing its capability to handle complex textual inputs. Additionally, an attention mechanism highlights semantically important features, further improving feature representation and classification performance. To evaluate the effectiveness of the proposed model, we conducted comparative experiments on the Amazon, IMDB, SST-2, and Ceramic datasets. The F1 scores achieved were 94.25%, 92.52%, 91.51%, and 81.32%, respectively, substantially outperforming existing models, particularly in handling domain-specific and nuanced texts. These results demonstrate that the proposed multi-feature fusion model significantly enhances sentiment classification performance.https://ieeexplore.ieee.org/document/11031473/RoBERTafeature extractiondeep learningattention mechanismsentiment analysis |
| spellingShingle | LiHua Yang Jun Wang WangRen Qiu RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis IEEE Access RoBERTa feature extraction deep learning attention mechanism sentiment analysis |
| title | RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis |
| title_full | RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis |
| title_fullStr | RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis |
| title_full_unstemmed | RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis |
| title_short | RoBERTa-Based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis |
| title_sort | roberta based multi feature integrated bilstm and cnn model for ceramic review analysis |
| topic | RoBERTa feature extraction deep learning attention mechanism sentiment analysis |
| url | https://ieeexplore.ieee.org/document/11031473/ |
| work_keys_str_mv | AT lihuayang robertabasedmultifeatureintegratedbilstmandcnnmodelforceramicreviewanalysis AT junwang robertabasedmultifeatureintegratedbilstmandcnnmodelforceramicreviewanalysis AT wangrenqiu robertabasedmultifeatureintegratedbilstmandcnnmodelforceramicreviewanalysis |