A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model
Physical fatigue significantly impacts safety and performance across industrial, athletic, and medical domains, yet its detection remains challenging due to individual variability and limited generalizability of existing methods. This study introduces a multimodal fatigue detection system integratin...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/11/3309 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849330979050618880 |
|---|---|
| author | Soree Hwang Nayeon Kwon Dongwon Lee Jongman Kim Sumin Yang Inchan Youn Hyuk-June Moon Joon-Kyung Sung Sungmin Han |
| author_facet | Soree Hwang Nayeon Kwon Dongwon Lee Jongman Kim Sumin Yang Inchan Youn Hyuk-June Moon Joon-Kyung Sung Sungmin Han |
| author_sort | Soree Hwang |
| collection | DOAJ |
| description | Physical fatigue significantly impacts safety and performance across industrial, athletic, and medical domains, yet its detection remains challenging due to individual variability and limited generalizability of existing methods. This study introduces a multimodal fatigue detection system integrating surface electromyography (sEMG) and inertial measurement unit (IMU) signals, processed through a hybrid convolutional neural network–long short-term memory–attention (CNN-LSTM-Attention) model. Fatigue was induced in 35 healthy participants via step-up-and-down exercises, with gait data collected during natural walking before and after fatigue. The model leverages sEMG from the gastrocnemius lateralis and IMU-derived jerk signals from the tibialis anterior and rectus femoris to classify fatigue states. Evaluated using leave-one-subject-out cross-validation (LOSOCV), the system achieved an accuracy of 87.94% with bilateral EMG signals and a balanced recall of 87.94% for fatigued states using a combined IMU-EMG approach. These results highlight the system’s robustness for personalized fatigue monitoring, surpassing traditional subject-dependent methods by addressing inter-individual differences. |
| format | Article |
| id | doaj-art-b047e1f9ea3d4e6387cd88e0285ffe5d |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-b047e1f9ea3d4e6387cd88e0285ffe5d2025-08-20T03:46:46ZengMDPI AGSensors1424-82202025-05-012511330910.3390/s25113309A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention ModelSoree Hwang0Nayeon Kwon1Dongwon Lee2Jongman Kim3Sumin Yang4Inchan Youn5Hyuk-June Moon6Joon-Kyung Sung7Sungmin Han8Bionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaSchool of Biomedical Engineering, Korea University, Seoul 02841, Republic of KoreaBionics Research Center, Biomedical Research Division, Korea Institute of Science and Technology (KIST), Seoul 02792, Republic of KoreaPhysical fatigue significantly impacts safety and performance across industrial, athletic, and medical domains, yet its detection remains challenging due to individual variability and limited generalizability of existing methods. This study introduces a multimodal fatigue detection system integrating surface electromyography (sEMG) and inertial measurement unit (IMU) signals, processed through a hybrid convolutional neural network–long short-term memory–attention (CNN-LSTM-Attention) model. Fatigue was induced in 35 healthy participants via step-up-and-down exercises, with gait data collected during natural walking before and after fatigue. The model leverages sEMG from the gastrocnemius lateralis and IMU-derived jerk signals from the tibialis anterior and rectus femoris to classify fatigue states. Evaluated using leave-one-subject-out cross-validation (LOSOCV), the system achieved an accuracy of 87.94% with bilateral EMG signals and a balanced recall of 87.94% for fatigued states using a combined IMU-EMG approach. These results highlight the system’s robustness for personalized fatigue monitoring, surpassing traditional subject-dependent methods by addressing inter-individual differences.https://www.mdpi.com/1424-8220/25/11/3309physical fatigue detectionsurface electromyography (sEMG)inertial measurement unit (IMU)hybrid deep learningCNN-LSTM-attentiongait kinematics |
| spellingShingle | Soree Hwang Nayeon Kwon Dongwon Lee Jongman Kim Sumin Yang Inchan Youn Hyuk-June Moon Joon-Kyung Sung Sungmin Han A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model Sensors physical fatigue detection surface electromyography (sEMG) inertial measurement unit (IMU) hybrid deep learning CNN-LSTM-attention gait kinematics |
| title | A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model |
| title_full | A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model |
| title_fullStr | A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model |
| title_full_unstemmed | A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model |
| title_short | A Multimodal Fatigue Detection System Using sEMG and IMU Signals with a Hybrid CNN-LSTM-Attention Model |
| title_sort | multimodal fatigue detection system using semg and imu signals with a hybrid cnn lstm attention model |
| topic | physical fatigue detection surface electromyography (sEMG) inertial measurement unit (IMU) hybrid deep learning CNN-LSTM-attention gait kinematics |
| url | https://www.mdpi.com/1424-8220/25/11/3309 |
| work_keys_str_mv | AT soreehwang amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT nayeonkwon amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT dongwonlee amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT jongmankim amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT suminyang amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT inchanyoun amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT hyukjunemoon amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT joonkyungsung amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT sungminhan amultimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT soreehwang multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT nayeonkwon multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT dongwonlee multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT jongmankim multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT suminyang multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT inchanyoun multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT hyukjunemoon multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT joonkyungsung multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel AT sungminhan multimodalfatiguedetectionsystemusingsemgandimusignalswithahybridcnnlstmattentionmodel |