Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring
Modern mental fatigue detection methods include many parameters for evaluation. For example, many researchers use human subjective evaluation or driving parameters to assess this human condition. Development of a method for detecting the functional state of mental fatigue is an extremely important t...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/21/6805 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846173167852191744 |
|---|---|
| author | Alexey Kashevnik Svetlana Kovalenko Anton Mamonov Batol Hamoud Aleksandr Bulygin Vladislav Kuznetsov Irina Shoshina Ivan Brak Gleb Kiselev |
| author_facet | Alexey Kashevnik Svetlana Kovalenko Anton Mamonov Batol Hamoud Aleksandr Bulygin Vladislav Kuznetsov Irina Shoshina Ivan Brak Gleb Kiselev |
| author_sort | Alexey Kashevnik |
| collection | DOAJ |
| description | Modern mental fatigue detection methods include many parameters for evaluation. For example, many researchers use human subjective evaluation or driving parameters to assess this human condition. Development of a method for detecting the functional state of mental fatigue is an extremely important task. Despite the fact that human operator support systems are becoming more and more widespread, at the moment there is no open-source solution that can monitor this human state based on eye movement monitoring in real time and with high accuracy. Such a method allows the prevention of a large number of potential hazardous situations and accidents in critical industries (nuclear stations, transport systems, and air traffic control). This paper describes the developed method for mental fatigue detection based on human eye movements. We based our research on a developed earlier dataset that included captured eye-tracking data of human operators that implemented different tasks during the day. In the scope of the developed method, we propose a technique for the determination of the most relevant gaze characteristics for mental fatigue state detection. The developed method includes the following machine learning techniques for human state classification: random forest, decision tree, and multilayered perceptron. The experimental results showed that the most relevant characteristics are as follows: average velocity within the fixation area; average curvature of the gaze trajectory; minimum curvature of the gaze trajectory; minimum saccade length; percentage of fixations shorter than 150 ms; and proportion of time spent in fixations shorter than 150 milliseconds. The processing of eye movement data using the proposed method is performed in real time, with the maximum accuracy (0.85) and F1-score (0.80) reached using the random forest method. |
| format | Article |
| id | doaj-art-6271b81d1fc4469ca920ac9f12b499b7 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-6271b81d1fc4469ca920ac9f12b499b72024-11-08T14:41:00ZengMDPI AGSensors1424-82202024-10-012421680510.3390/s24216805Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement MonitoringAlexey Kashevnik0Svetlana Kovalenko1Anton Mamonov2Batol Hamoud3Aleksandr Bulygin4Vladislav Kuznetsov5Irina Shoshina6Ivan Brak7Gleb Kiselev8St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg 199178, RussiaLaboratory for Cognitive Psychology of Digital Interface Users, HSE University, Moscow 101000, RussiaFaculty of Physics and Mathematics and Natural Sciences, Peoples’ Friendship University of Russia, Moscow 117198, RussiaSt. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg 199178, RussiaSt. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg 199178, RussiaFederal Research Center “Computer Science and Control” of Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, RussiaInstitute for Cognitive Research, Saint Petersburg State University, St. Petersburg 199034, RussiaFaculty of Physics and Mathematics and Natural Sciences, Peoples’ Friendship University of Russia, Moscow 117198, RussiaFaculty of Physics and Mathematics and Natural Sciences, Peoples’ Friendship University of Russia, Moscow 117198, RussiaModern mental fatigue detection methods include many parameters for evaluation. For example, many researchers use human subjective evaluation or driving parameters to assess this human condition. Development of a method for detecting the functional state of mental fatigue is an extremely important task. Despite the fact that human operator support systems are becoming more and more widespread, at the moment there is no open-source solution that can monitor this human state based on eye movement monitoring in real time and with high accuracy. Such a method allows the prevention of a large number of potential hazardous situations and accidents in critical industries (nuclear stations, transport systems, and air traffic control). This paper describes the developed method for mental fatigue detection based on human eye movements. We based our research on a developed earlier dataset that included captured eye-tracking data of human operators that implemented different tasks during the day. In the scope of the developed method, we propose a technique for the determination of the most relevant gaze characteristics for mental fatigue state detection. The developed method includes the following machine learning techniques for human state classification: random forest, decision tree, and multilayered perceptron. The experimental results showed that the most relevant characteristics are as follows: average velocity within the fixation area; average curvature of the gaze trajectory; minimum curvature of the gaze trajectory; minimum saccade length; percentage of fixations shorter than 150 ms; and proportion of time spent in fixations shorter than 150 milliseconds. The processing of eye movement data using the proposed method is performed in real time, with the maximum accuracy (0.85) and F1-score (0.80) reached using the random forest method.https://www.mdpi.com/1424-8220/24/21/6805mental fatigue detectioneye-trackingmachine learning |
| spellingShingle | Alexey Kashevnik Svetlana Kovalenko Anton Mamonov Batol Hamoud Aleksandr Bulygin Vladislav Kuznetsov Irina Shoshina Ivan Brak Gleb Kiselev Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring Sensors mental fatigue detection eye-tracking machine learning |
| title | Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring |
| title_full | Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring |
| title_fullStr | Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring |
| title_full_unstemmed | Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring |
| title_short | Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring |
| title_sort | intelligent human operator mental fatigue assessment method based on gaze movement monitoring |
| topic | mental fatigue detection eye-tracking machine learning |
| url | https://www.mdpi.com/1424-8220/24/21/6805 |
| work_keys_str_mv | AT alexeykashevnik intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT svetlanakovalenko intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT antonmamonov intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT batolhamoud intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT aleksandrbulygin intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT vladislavkuznetsov intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT irinashoshina intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT ivanbrak intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring AT glebkiselev intelligenthumanoperatormentalfatigueassessmentmethodbasedongazemovementmonitoring |