ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction
Advanced human-machine interaction (AHMI) is a key concept in human factors and ergonomics (HFE), focusing on how individuals interact with systems to perform tasks efficiently. As AHMI becomes more integrated into fields such as Industry 4.0, aviation, automotive, and clinical applications, users f...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11078285/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850077348777426944 |
|---|---|
| author | Gabriele Luzzani Marco Pogliano Irene Buraioli Manuel Colavincenzo Stefano Martorana Giorgio Guglieri Danilo Demarchi |
| author_facet | Gabriele Luzzani Marco Pogliano Irene Buraioli Manuel Colavincenzo Stefano Martorana Giorgio Guglieri Danilo Demarchi |
| author_sort | Gabriele Luzzani |
| collection | DOAJ |
| description | Advanced human-machine interaction (AHMI) is a key concept in human factors and ergonomics (HFE), focusing on how individuals interact with systems to perform tasks efficiently. As AHMI becomes more integrated into fields such as Industry 4.0, aviation, automotive, and clinical applications, users face increasing complexity, leading to elevated mental workload (MWL) and stress. These factors can impair performance and cause accidents, emphasizing the need for AHMI systems capable of real-time monitoring of cognitive load and stress levels. This paper investigates the relationship between stress, MWL, and four physiological signals—electrocardiogram (ECG), respiration, functional near-infrared spectroscopy (fNIRS), and eye tracking—combined with a tailored Self-Assessment Questionnaire (SAQ), specifically designed for industrial applications. A study involving 20 participants was conducted using the Stroop, Visual, Auditory, and Dual N-Back tasks. During the study, 83 features were extracted from the physiological signals and linked to this four-level ratings SAQ of perceived stress and MWL. Statistical analysis using Kruskal-Wallis and Mann-Whitney tests assessed the ability of these features to differentiate stress and MWL levels. Over 50% of the features reliably distinguished between cognitive states, particularly in identifying relaxed versus altered conditions. Respiration, fNIRS, and eye movement signals provided higher granularity in differentiating multiple altered cognitive states, suggesting their potential for precise monitoring in AHMI systems. These findings underscore the value of physiological monitoring in AHMI systems, which can enhance user performance and safety by enabling adaptive interfaces tailored to real-time cognitive states, supporting future industrial applications. |
| format | Article |
| id | doaj-art-a8f2d0b328924576beda91ce82ae1e1b |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-a8f2d0b328924576beda91ce82ae1e1b2025-08-20T02:45:49ZengIEEEIEEE Access2169-35362025-01-011312272612274110.1109/ACCESS.2025.358838411078285ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine InteractionGabriele Luzzani0https://orcid.org/0000-0002-6000-7816Marco Pogliano1https://orcid.org/0009-0008-3187-3193Irene Buraioli2https://orcid.org/0000-0002-5419-7772Manuel Colavincenzo3Stefano Martorana4Giorgio Guglieri5https://orcid.org/0000-0001-6455-5062Danilo Demarchi6https://orcid.org/0000-0001-5374-1679Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, ItalyDepartment of Electronics and Telecommunication, Politecnico di Torino, Turin, ItalyDepartment of Electronics and Telecommunication, Politecnico di Torino, Turin, ItalyLeonardo Innovation Labs, Turin, ItalyAircraft Division, Leonardo S.p.a., Turin, ItalyDepartment of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, ItalyDepartment of Electronics and Telecommunication, Politecnico di Torino, Turin, ItalyAdvanced human-machine interaction (AHMI) is a key concept in human factors and ergonomics (HFE), focusing on how individuals interact with systems to perform tasks efficiently. As AHMI becomes more integrated into fields such as Industry 4.0, aviation, automotive, and clinical applications, users face increasing complexity, leading to elevated mental workload (MWL) and stress. These factors can impair performance and cause accidents, emphasizing the need for AHMI systems capable of real-time monitoring of cognitive load and stress levels. This paper investigates the relationship between stress, MWL, and four physiological signals—electrocardiogram (ECG), respiration, functional near-infrared spectroscopy (fNIRS), and eye tracking—combined with a tailored Self-Assessment Questionnaire (SAQ), specifically designed for industrial applications. A study involving 20 participants was conducted using the Stroop, Visual, Auditory, and Dual N-Back tasks. During the study, 83 features were extracted from the physiological signals and linked to this four-level ratings SAQ of perceived stress and MWL. Statistical analysis using Kruskal-Wallis and Mann-Whitney tests assessed the ability of these features to differentiate stress and MWL levels. Over 50% of the features reliably distinguished between cognitive states, particularly in identifying relaxed versus altered conditions. Respiration, fNIRS, and eye movement signals provided higher granularity in differentiating multiple altered cognitive states, suggesting their potential for precise monitoring in AHMI systems. These findings underscore the value of physiological monitoring in AHMI systems, which can enhance user performance and safety by enabling adaptive interfaces tailored to real-time cognitive states, supporting future industrial applications.https://ieeexplore.ieee.org/document/11078285/ECGeye trackingfNIRSrespirationmental workloadstress |
| spellingShingle | Gabriele Luzzani Marco Pogliano Irene Buraioli Manuel Colavincenzo Stefano Martorana Giorgio Guglieri Danilo Demarchi ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction IEEE Access ECG eye tracking fNIRS respiration mental workload stress |
| title | ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction |
| title_full | ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction |
| title_fullStr | ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction |
| title_full_unstemmed | ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction |
| title_short | ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction |
| title_sort | ecg respiration fnirs and eye tracking for stress and mental workload monitoring in human machine interaction |
| topic | ECG eye tracking fNIRS respiration mental workload stress |
| url | https://ieeexplore.ieee.org/document/11078285/ |
| work_keys_str_mv | AT gabrieleluzzani ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction AT marcopogliano ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction AT ireneburaioli ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction AT manuelcolavincenzo ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction AT stefanomartorana ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction AT giorgioguglieri ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction AT danilodemarchi ecgrespirationfnirsandeyetrackingforstressandmentalworkloadmonitoringinhumanmachineinteraction |