Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer
Epilepsy has been analyzed through uni-modality non-invasive brain measurements such as electroencephalogram (EEG) signal, but identifying seizure patterns is more challenging due to the non-stationary nature of the brain activity and various non-brain artifacts. In this article, we leverage a visio...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Open Journal of the Computer Society |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10755173/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846152018355290112 |
|---|---|
| author | Rafat Damseh Abdelhadi Hireche Parikshat Sirpal Abdelkader Nasreddine Belkacem |
| author_facet | Rafat Damseh Abdelhadi Hireche Parikshat Sirpal Abdelkader Nasreddine Belkacem |
| author_sort | Rafat Damseh |
| collection | DOAJ |
| description | Epilepsy has been analyzed through uni-modality non-invasive brain measurements such as electroencephalogram (EEG) signal, but identifying seizure patterns is more challenging due to the non-stationary nature of the brain activity and various non-brain artifacts. In this article, we leverage a vision transformer model (ViT) to classify three types of seizure patterns based on multimodal EEG and functional near-infrared spectroscopy (fNIRS) recordings. We used spectral encoding techniques to capture temporal and spatial relationships for brain signals as feature map inputs to the transformer architecture. We evaluated model performance using the receiver operating characteristic (ROC) curves and the area under the curve (AUC), demonstrating that multimodal EEG-fNIRS signals improved the classification accuracy of seizure patterns. Our work showed that power spectral density (PSD) features often led to better results than features derived from dynamic mode decomposition (DMD), particularly for seizures with high-frequency oscillations (HFO) and generalized spike-and-wave discharge (GSWD) patterns, with an accuracy of 93.14% and 91.69%, respectively. Low-voltage fast activity (LVFA) seizures achieved consistently high performance in EEG, fNIRS, and multimodal EEG-fNIRS setups. Overall, our findings suggest the effectiveness of using the ViT architecture with multimodal brain data accompanied by appropriate spectral features to classify the neural activity of epileptic seizure patterns. |
| format | Article |
| id | doaj-art-f5305dbe62df47a08b8ed81073e2bd1c |
| institution | Kabale University |
| issn | 2644-1268 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of the Computer Society |
| spelling | doaj-art-f5305dbe62df47a08b8ed81073e2bd1c2024-11-27T00:00:59ZengIEEEIEEE Open Journal of the Computer Society2644-12682024-01-01572473510.1109/OJCS.2024.350003210755173Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision TransformerRafat Damseh0https://orcid.org/0000-0001-6797-0448Abdelhadi Hireche1https://orcid.org/0009-0000-0633-9281Parikshat Sirpal2https://orcid.org/0000-0002-9047-3439Abdelkader Nasreddine Belkacem3https://orcid.org/0000-0002-3024-4167Department of Computer Science and Software Engineering, United Arab Emirates University, Al Ain Abu Dhabi, United Arab EmiratesDepartment of Computer and Network Engineering, United Arab Emirates University, Al Ain Abu Dhabi, United Arab EmiratesSchool of Electrical and Computer Engineering, Gallogly College of Engineering, University of Oklahoma, Norman, OK, USADepartment of Computer and Network Engineering, United Arab Emirates University, Al Ain Abu Dhabi, United Arab EmiratesEpilepsy has been analyzed through uni-modality non-invasive brain measurements such as electroencephalogram (EEG) signal, but identifying seizure patterns is more challenging due to the non-stationary nature of the brain activity and various non-brain artifacts. In this article, we leverage a vision transformer model (ViT) to classify three types of seizure patterns based on multimodal EEG and functional near-infrared spectroscopy (fNIRS) recordings. We used spectral encoding techniques to capture temporal and spatial relationships for brain signals as feature map inputs to the transformer architecture. We evaluated model performance using the receiver operating characteristic (ROC) curves and the area under the curve (AUC), demonstrating that multimodal EEG-fNIRS signals improved the classification accuracy of seizure patterns. Our work showed that power spectral density (PSD) features often led to better results than features derived from dynamic mode decomposition (DMD), particularly for seizures with high-frequency oscillations (HFO) and generalized spike-and-wave discharge (GSWD) patterns, with an accuracy of 93.14% and 91.69%, respectively. Low-voltage fast activity (LVFA) seizures achieved consistently high performance in EEG, fNIRS, and multimodal EEG-fNIRS setups. Overall, our findings suggest the effectiveness of using the ViT architecture with multimodal brain data accompanied by appropriate spectral features to classify the neural activity of epileptic seizure patterns.https://ieeexplore.ieee.org/document/10755173/Electroencephalogram (EEG)functional near-infrared spectroscopy (fNIRS)low-voltage fast activity (LVFA)generalized spike-and-wave discharge (GSWD)high-frequency oscillation (HFO)epilepsy |
| spellingShingle | Rafat Damseh Abdelhadi Hireche Parikshat Sirpal Abdelkader Nasreddine Belkacem Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer IEEE Open Journal of the Computer Society Electroencephalogram (EEG) functional near-infrared spectroscopy (fNIRS) low-voltage fast activity (LVFA) generalized spike-and-wave discharge (GSWD) high-frequency oscillation (HFO) epilepsy |
| title | Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer |
| title_full | Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer |
| title_fullStr | Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer |
| title_full_unstemmed | Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer |
| title_short | Multimodal EEG-fNIRS Seizure Pattern Decoding Using Vision Transformer |
| title_sort | multimodal eeg fnirs seizure pattern decoding using vision transformer |
| topic | Electroencephalogram (EEG) functional near-infrared spectroscopy (fNIRS) low-voltage fast activity (LVFA) generalized spike-and-wave discharge (GSWD) high-frequency oscillation (HFO) epilepsy |
| url | https://ieeexplore.ieee.org/document/10755173/ |
| work_keys_str_mv | AT rafatdamseh multimodaleegfnirsseizurepatterndecodingusingvisiontransformer AT abdelhadihireche multimodaleegfnirsseizurepatterndecodingusingvisiontransformer AT parikshatsirpal multimodaleegfnirsseizurepatterndecodingusingvisiontransformer AT abdelkadernasreddinebelkacem multimodaleegfnirsseizurepatterndecodingusingvisiontransformer |