A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects
Abstract EEG-based emotion recognition uses high-level information from neural activities to predict emotional responses in subjects. However, this information is sparsely distributed in frequency, time, and spatial domains and varied across subjects. To address these challenges in emotion recogniti...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2024-12-01
|
| Series: | Brain Informatics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40708-024-00242-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850134452902035456 |
|---|---|
| author | Rui Li Xuanwen Yang Jun Lou Junsong Zhang |
| author_facet | Rui Li Xuanwen Yang Jun Lou Junsong Zhang |
| author_sort | Rui Li |
| collection | DOAJ |
| description | Abstract EEG-based emotion recognition uses high-level information from neural activities to predict emotional responses in subjects. However, this information is sparsely distributed in frequency, time, and spatial domains and varied across subjects. To address these challenges in emotion recognition, we propose a novel neural network model named Temporal-Spectral Graph Convolutional Network (TSGCN). To capture high-level information distributed in time, spatial, and frequency domains, TSGCN considers both neural oscillation changes in different time windows and topological structures between different brain regions. Specifically, a Minimum Category Confusion (MCC) loss is used in TSGCN to reduce the inconsistencies between subjective ratings and predefined labels. In addition, to improve the generalization of TSGCN on cross-subject variation, we propose Deep and Shallow feature Dynamic Adversarial Learning (DSDAL) to calculate the distance between the source domain and the target domain. Extensive experiments were conducted on public datasets to demonstrate that TSGCN outperforms state-of-the-art methods in EEG-based emotion recognition. Ablation studies show that the mixed neural networks and our proposed methods in TSGCN significantly contribute to its high performance and robustness. Detailed investigations further provide the effectiveness of TSGCN in addressing the challenges in emotion recognition. |
| format | Article |
| id | doaj-art-1ed1c21df0094643b1a49ee17798f00a |
| institution | OA Journals |
| issn | 2198-4018 2198-4026 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Brain Informatics |
| spelling | doaj-art-1ed1c21df0094643b1a49ee17798f00a2025-08-20T02:31:42ZengSpringerOpenBrain Informatics2198-40182198-40262024-12-0111111810.1186/s40708-024-00242-xA temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjectsRui Li0Xuanwen Yang1Jun Lou2Junsong Zhang3Brain Cognition and Computing Lab, National Engineering Research Center for E-Learning, Faculty of Artificial Intelligence in Education, Central China Normal UniversityBrain Cognition and Computing Lab, National Engineering Research Center for E-Learning, Faculty of Artificial Intelligence in Education, Central China Normal UniversityBrain Cognition and Computing Lab, National Engineering Research Center for E-Learning, Faculty of Artificial Intelligence in Education, Central China Normal UniversityBrain Cognition and Intelligent Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen UniversityAbstract EEG-based emotion recognition uses high-level information from neural activities to predict emotional responses in subjects. However, this information is sparsely distributed in frequency, time, and spatial domains and varied across subjects. To address these challenges in emotion recognition, we propose a novel neural network model named Temporal-Spectral Graph Convolutional Network (TSGCN). To capture high-level information distributed in time, spatial, and frequency domains, TSGCN considers both neural oscillation changes in different time windows and topological structures between different brain regions. Specifically, a Minimum Category Confusion (MCC) loss is used in TSGCN to reduce the inconsistencies between subjective ratings and predefined labels. In addition, to improve the generalization of TSGCN on cross-subject variation, we propose Deep and Shallow feature Dynamic Adversarial Learning (DSDAL) to calculate the distance between the source domain and the target domain. Extensive experiments were conducted on public datasets to demonstrate that TSGCN outperforms state-of-the-art methods in EEG-based emotion recognition. Ablation studies show that the mixed neural networks and our proposed methods in TSGCN significantly contribute to its high performance and robustness. Detailed investigations further provide the effectiveness of TSGCN in addressing the challenges in emotion recognition.https://doi.org/10.1186/s40708-024-00242-xEEG signalAffective computingGraph neural networkCross-subjects |
| spellingShingle | Rui Li Xuanwen Yang Jun Lou Junsong Zhang A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects Brain Informatics EEG signal Affective computing Graph neural network Cross-subjects |
| title | A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects |
| title_full | A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects |
| title_fullStr | A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects |
| title_full_unstemmed | A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects |
| title_short | A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects |
| title_sort | temporal spectral graph convolutional neural network model for eeg emotion recognition within and across subjects |
| topic | EEG signal Affective computing Graph neural network Cross-subjects |
| url | https://doi.org/10.1186/s40708-024-00242-x |
| work_keys_str_mv | AT ruili atemporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT xuanwenyang atemporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT junlou atemporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT junsongzhang atemporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT ruili temporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT xuanwenyang temporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT junlou temporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects AT junsongzhang temporalspectralgraphconvolutionalneuralnetworkmodelforeegemotionrecognitionwithinandacrosssubjects |