Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion
Fatigue and distracted driving are two of the leading causes of major accidents. Drivers play a crucial role in automobile safety, and accurately detecting their driving states can significantly enhance the safety of urban road traffic and improve road operation efficiency. Currently, mainstream res...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10994423/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850271519565938688 |
|---|---|
| author | Deyong Guan Qi Wang Ke Wang Xinyu Song |
| author_facet | Deyong Guan Qi Wang Ke Wang Xinyu Song |
| author_sort | Deyong Guan |
| collection | DOAJ |
| description | Fatigue and distracted driving are two of the leading causes of major accidents. Drivers play a crucial role in automobile safety, and accurately detecting their driving states can significantly enhance the safety of urban road traffic and improve road operation efficiency. Currently, mainstream research relies on visual features, EEG (Electroencephalography), EOG(Electrooculography), and EMG (Electromyography) to identify driving status. However, these methods are not easily scalable due to the high cost of the data acquisition devices and the fact that physical contact with the body can interfere with normal driving functions. This study conducted a simulation experiment using a driving simulator and physiological wristbands, with a highway as the experimental scenario. Data on the physiological and psychological parameters, driving performance, and wrist movement information of 33 drivers were collected. Firstly, features were extracted through time, frequency domain, and nonlinear analyses, and non-parametric tests were performed to analyze the differences in these features under different driving states, thus selecting an effective feature subset. Secondly, based on this, an impaired driving state recognition model was established by combining feature recursive elimination and machine learning, and the impact of different modal inputs on the model performance was analyzed. Finally, SHAP (SHapley Additive exPlanations) interpretable machine learning was employed to analyze the model results in depth. The results indicate that the XGBoost model based on multimodal information input performs the best, with accuracy, precision, F1 score, and recall reaching 94.28%, 94.28%, 94%, and 94.27%, respectively, demonstrating its effectiveness in recognizing driving status. The mean X-axis angular velocity of wrist motion is a key feature for identifying a driver’s driving status. Additionally, the mean X-axis angular velocity, mean lane offset, and mean X-axis acceleration are positively correlated with the probability of fatigued driving, while the standard deviation of yaw rate is positively correlated with the likelihood of a distracted driving state. This study is the first to use a fusion of physiological data, driving performance, and wrist movement information to identify impaired driving states. This method takes into account various impaired driving states and provides a non-invasive, robust, and real-time approach, which has practical significance. |
| format | Article |
| id | doaj-art-cb207c1e8c30422bb4dffe561116d6c3 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-cb207c1e8c30422bb4dffe561116d6c32025-08-20T01:52:12ZengIEEEIEEE Access2169-35362025-01-0113838158382710.1109/ACCESS.2025.356862510994423Fatigue and Distracted Driving Recognition Method Based on Multimodal Information FusionDeyong Guan0https://orcid.org/0000-0003-3258-6440Qi Wang1https://orcid.org/0009-0001-2239-5336Ke Wang2https://orcid.org/0000-0001-8941-723XXinyu Song3https://orcid.org/0009-0009-9117-8018College of Transportation, Shandong University of Science and Technology, Qingdao, Shandong, ChinaCollege of Transportation, Shandong University of Science and Technology, Qingdao, Shandong, ChinaCollege of Transportation, Shandong University of Science and Technology, Qingdao, Shandong, ChinaCollege of Transportation, Shandong University of Science and Technology, Qingdao, Shandong, ChinaFatigue and distracted driving are two of the leading causes of major accidents. Drivers play a crucial role in automobile safety, and accurately detecting their driving states can significantly enhance the safety of urban road traffic and improve road operation efficiency. Currently, mainstream research relies on visual features, EEG (Electroencephalography), EOG(Electrooculography), and EMG (Electromyography) to identify driving status. However, these methods are not easily scalable due to the high cost of the data acquisition devices and the fact that physical contact with the body can interfere with normal driving functions. This study conducted a simulation experiment using a driving simulator and physiological wristbands, with a highway as the experimental scenario. Data on the physiological and psychological parameters, driving performance, and wrist movement information of 33 drivers were collected. Firstly, features were extracted through time, frequency domain, and nonlinear analyses, and non-parametric tests were performed to analyze the differences in these features under different driving states, thus selecting an effective feature subset. Secondly, based on this, an impaired driving state recognition model was established by combining feature recursive elimination and machine learning, and the impact of different modal inputs on the model performance was analyzed. Finally, SHAP (SHapley Additive exPlanations) interpretable machine learning was employed to analyze the model results in depth. The results indicate that the XGBoost model based on multimodal information input performs the best, with accuracy, precision, F1 score, and recall reaching 94.28%, 94.28%, 94%, and 94.27%, respectively, demonstrating its effectiveness in recognizing driving status. The mean X-axis angular velocity of wrist motion is a key feature for identifying a driver’s driving status. Additionally, the mean X-axis angular velocity, mean lane offset, and mean X-axis acceleration are positively correlated with the probability of fatigued driving, while the standard deviation of yaw rate is positively correlated with the likelihood of a distracted driving state. This study is the first to use a fusion of physiological data, driving performance, and wrist movement information to identify impaired driving states. This method takes into account various impaired driving states and provides a non-invasive, robust, and real-time approach, which has practical significance.https://ieeexplore.ieee.org/document/10994423/Fatigued drivingdistracted drivingmultimodal information fusionXGBoostSHAP |
| spellingShingle | Deyong Guan Qi Wang Ke Wang Xinyu Song Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion IEEE Access Fatigued driving distracted driving multimodal information fusion XGBoost SHAP |
| title | Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion |
| title_full | Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion |
| title_fullStr | Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion |
| title_full_unstemmed | Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion |
| title_short | Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion |
| title_sort | fatigue and distracted driving recognition method based on multimodal information fusion |
| topic | Fatigued driving distracted driving multimodal information fusion XGBoost SHAP |
| url | https://ieeexplore.ieee.org/document/10994423/ |
| work_keys_str_mv | AT deyongguan fatigueanddistracteddrivingrecognitionmethodbasedonmultimodalinformationfusion AT qiwang fatigueanddistracteddrivingrecognitionmethodbasedonmultimodalinformationfusion AT kewang fatigueanddistracteddrivingrecognitionmethodbasedonmultimodalinformationfusion AT xinyusong fatigueanddistracteddrivingrecognitionmethodbasedonmultimodalinformationfusion |