MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots
Emotion recognition based on electroencephalography (EEG) signals has garnered significant research attention in recent years due to its potential applications in affective computing and brain-computer interfaces. Despite the proposal of various deep learning-based methods for extracting emotional f...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11095664/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849245972130955264 |
|---|---|
| author | Zihan Zhang Zhiyong Zhou Jun Wang Hao Hu Jing Zhao |
| author_facet | Zihan Zhang Zhiyong Zhou Jun Wang Hao Hu Jing Zhao |
| author_sort | Zihan Zhang |
| collection | DOAJ |
| description | Emotion recognition based on electroencephalography (EEG) signals has garnered significant research attention in recent years due to its potential applications in affective computing and brain-computer interfaces. Despite the proposal of various deep learning-based methods for extracting emotional features from EEG signals, most existing models struggle to effectively capture both long-term and short-term dependencies within the signals, failing to fully integrate features across different temporal scales. To address these challenges, we propose a deep learning model that combines multi-temporal-scale fusion, termed MT-EfficientNetV2. This model segments one-dimensional EEG signals using combinations of varying window sizes and fixed step lengths. The Recursive Plot (RP) algorithm is then employed to transform these segments into RGB images that intuitively represent the dynamic characteristics of the signals, facilitating the capture of complex emotional features. Additionally, a three-branch input feature fusion module has been designed to effectively integrate features across different scales within the same temporal domain. The model architecture incorporates DEconv and the SimAM attention mechanism with EfficientNetV2. This integration enhances the global fusion and expression of multi-scale features while strengthening the extraction of key emotional features at the local level, thereby suppressing redundant information. Experiments conducted on the public datasets SEED and SEED-IV yielded accuracies of 98.67% and 96.89%, respectively, surpassing current mainstream methods and validating the feasibility and effectiveness of the proposed approach. |
| format | Article |
| id | doaj-art-8a3c948f1dca4c2dbd002f47bc77f03c |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-8a3c948f1dca4c2dbd002f47bc77f03c2025-08-20T03:58:39ZengIEEEIEEE Access2169-35362025-01-011313207913209610.1109/ACCESS.2025.359233611095664MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence PlotsZihan Zhang0https://orcid.org/0009-0000-3713-2435Zhiyong Zhou1https://orcid.org/0009-0006-9046-3333Jun Wang2https://orcid.org/0000-0001-9115-3755Hao Hu3https://orcid.org/0009-0000-5557-2266Jing Zhao4School of Electronic Information Engineering, Shanghai Dianji University, Shanghai, ChinaSchool of Art and Design, Shanghai Dianji University, Shanghai, ChinaSchool of Computer and Computing Science, Hangzhou City University, Hangzhou, ChinaSchool of Mechanical, Shanghai Dianji University, Shanghai, ChinaSchool of Electronic Information Engineering, Shanghai Dianji University, Shanghai, ChinaEmotion recognition based on electroencephalography (EEG) signals has garnered significant research attention in recent years due to its potential applications in affective computing and brain-computer interfaces. Despite the proposal of various deep learning-based methods for extracting emotional features from EEG signals, most existing models struggle to effectively capture both long-term and short-term dependencies within the signals, failing to fully integrate features across different temporal scales. To address these challenges, we propose a deep learning model that combines multi-temporal-scale fusion, termed MT-EfficientNetV2. This model segments one-dimensional EEG signals using combinations of varying window sizes and fixed step lengths. The Recursive Plot (RP) algorithm is then employed to transform these segments into RGB images that intuitively represent the dynamic characteristics of the signals, facilitating the capture of complex emotional features. Additionally, a three-branch input feature fusion module has been designed to effectively integrate features across different scales within the same temporal domain. The model architecture incorporates DEconv and the SimAM attention mechanism with EfficientNetV2. This integration enhances the global fusion and expression of multi-scale features while strengthening the extraction of key emotional features at the local level, thereby suppressing redundant information. Experiments conducted on the public datasets SEED and SEED-IV yielded accuracies of 98.67% and 96.89%, respectively, surpassing current mainstream methods and validating the feasibility and effectiveness of the proposed approach.https://ieeexplore.ieee.org/document/11095664/EEGemotion recognitionaffective computingrecursive plotDEconvSimAM attention mechanism |
| spellingShingle | Zihan Zhang Zhiyong Zhou Jun Wang Hao Hu Jing Zhao MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots IEEE Access EEG emotion recognition affective computing recursive plot DEconv SimAM attention mechanism |
| title | MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots |
| title_full | MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots |
| title_fullStr | MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots |
| title_full_unstemmed | MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots |
| title_short | MT-EfficientNetV2: A Multi-Temporal Scale Fusion EEG Emotion Recognition Method Based on Recurrence Plots |
| title_sort | mt efficientnetv2 a multi temporal scale fusion eeg emotion recognition method based on recurrence plots |
| topic | EEG emotion recognition affective computing recursive plot DEconv SimAM attention mechanism |
| url | https://ieeexplore.ieee.org/document/11095664/ |
| work_keys_str_mv | AT zihanzhang mtefficientnetv2amultitemporalscalefusioneegemotionrecognitionmethodbasedonrecurrenceplots AT zhiyongzhou mtefficientnetv2amultitemporalscalefusioneegemotionrecognitionmethodbasedonrecurrenceplots AT junwang mtefficientnetv2amultitemporalscalefusioneegemotionrecognitionmethodbasedonrecurrenceplots AT haohu mtefficientnetv2amultitemporalscalefusioneegemotionrecognitionmethodbasedonrecurrenceplots AT jingzhao mtefficientnetv2amultitemporalscalefusioneegemotionrecognitionmethodbasedonrecurrenceplots |