Cross-modality fusion with EEG and text for enhanced emotion detection in English writing
IntroductionEmotion detection in written text is critical for applications in human-computer interaction, affective computing, and personalized content recommendation. Traditional approaches to emotion detection primarily leverage textual features, using natural language processing techniques such a...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2024.1529880/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850024382860099584 |
|---|---|
| author | Jing Wang Ci Zhang |
| author_facet | Jing Wang Ci Zhang |
| author_sort | Jing Wang |
| collection | DOAJ |
| description | IntroductionEmotion detection in written text is critical for applications in human-computer interaction, affective computing, and personalized content recommendation. Traditional approaches to emotion detection primarily leverage textual features, using natural language processing techniques such as sentiment analysis, which, while effective, may miss subtle nuances of emotions. These methods often fall short in recognizing the complex, multimodal nature of human emotions, as they ignore physiological cues that could provide richer emotional insights.MethodsTo address these limitations, this paper proposes Emotion Fusion-Transformer, a cross-modality fusion model that integrates EEG signals and textual data to enhance emotion detection in English writing. By utilizing the Transformer architecture, our model effectively captures contextual relationships within the text while concurrently processing EEG signals to extract underlying emotional states. Specifically, the Emotion Fusion-Transformer first preprocesses EEG data through signal transformation and filtering, followed by feature extraction that complements the textual embeddings. These modalities are fused within a unified Transformer framework, allowing for a holistic view of both the cognitive and physiological dimensions of emotion.Results and discussionExperimental results demonstrate that the proposed model significantly outperforms text-only and EEG-only approaches, with improvements in both accuracy and F1-score across diverse emotional categories. This model shows promise for enhancing affective computing applications by bridging the gap between physiological and textual emotion detection, enabling more nuanced and accurate emotion analysis in English writing. |
| format | Article |
| id | doaj-art-b533126444e746c3aa548b930304846b |
| institution | DOAJ |
| issn | 1662-5218 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj-art-b533126444e746c3aa548b930304846b2025-08-20T03:01:07ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182025-01-011810.3389/fnbot.2024.15298801529880Cross-modality fusion with EEG and text for enhanced emotion detection in English writingJing Wang0Ci Zhang1School of Foreign Languages, Henan Polytechnic University, Jiaozuo, ChinaCollege of Foreign Languages, Wenzhou University, Wenzhou, ChinaIntroductionEmotion detection in written text is critical for applications in human-computer interaction, affective computing, and personalized content recommendation. Traditional approaches to emotion detection primarily leverage textual features, using natural language processing techniques such as sentiment analysis, which, while effective, may miss subtle nuances of emotions. These methods often fall short in recognizing the complex, multimodal nature of human emotions, as they ignore physiological cues that could provide richer emotional insights.MethodsTo address these limitations, this paper proposes Emotion Fusion-Transformer, a cross-modality fusion model that integrates EEG signals and textual data to enhance emotion detection in English writing. By utilizing the Transformer architecture, our model effectively captures contextual relationships within the text while concurrently processing EEG signals to extract underlying emotional states. Specifically, the Emotion Fusion-Transformer first preprocesses EEG data through signal transformation and filtering, followed by feature extraction that complements the textual embeddings. These modalities are fused within a unified Transformer framework, allowing for a holistic view of both the cognitive and physiological dimensions of emotion.Results and discussionExperimental results demonstrate that the proposed model significantly outperforms text-only and EEG-only approaches, with improvements in both accuracy and F1-score across diverse emotional categories. This model shows promise for enhancing affective computing applications by bridging the gap between physiological and textual emotion detection, enabling more nuanced and accurate emotion analysis in English writing.https://www.frontiersin.org/articles/10.3389/fnbot.2024.1529880/fullemotion detectionEEGtextual analysistransformercross-modality fusion |
| spellingShingle | Jing Wang Ci Zhang Cross-modality fusion with EEG and text for enhanced emotion detection in English writing Frontiers in Neurorobotics emotion detection EEG textual analysis transformer cross-modality fusion |
| title | Cross-modality fusion with EEG and text for enhanced emotion detection in English writing |
| title_full | Cross-modality fusion with EEG and text for enhanced emotion detection in English writing |
| title_fullStr | Cross-modality fusion with EEG and text for enhanced emotion detection in English writing |
| title_full_unstemmed | Cross-modality fusion with EEG and text for enhanced emotion detection in English writing |
| title_short | Cross-modality fusion with EEG and text for enhanced emotion detection in English writing |
| title_sort | cross modality fusion with eeg and text for enhanced emotion detection in english writing |
| topic | emotion detection EEG textual analysis transformer cross-modality fusion |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2024.1529880/full |
| work_keys_str_mv | AT jingwang crossmodalityfusionwitheegandtextforenhancedemotiondetectioninenglishwriting AT cizhang crossmodalityfusionwitheegandtextforenhancedemotiondetectioninenglishwriting |