Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model
Background and objective: Background and objective: Human fatigue is a major cause of road traffic accidents. Currently widely used fatigue driving detection methods are based on eyelid closure, vehicle information or physiological parameter detection. However, the detection of each single feature h...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-11-01
|
| Series: | Heliyon |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2405844024155107 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846167022281424896 |
|---|---|
| author | Lu Yu Xinyi Yang Hengjian Wei Jianguo Liu Bo Li |
| author_facet | Lu Yu Xinyi Yang Hengjian Wei Jianguo Liu Bo Li |
| author_sort | Lu Yu |
| collection | DOAJ |
| description | Background and objective: Background and objective: Human fatigue is a major cause of road traffic accidents. Currently widely used fatigue driving detection methods are based on eyelid closure, vehicle information or physiological parameter detection. However, the detection of each single feature has certain limitations. Which in turn affects the accuracy of detection and the possibility and efficiency of prediction. Methods: This paper introduces a novel driver fatigue detection framework that leverages facial features, head pose, and PPG signals to establish a fatigue detection model. To validate this approach, a real-road driving experiment was conducted, resulting in the acquisition of multi-source feature signal data from 30 drivers. Utilizing a method for locating 68 facial landmarks, we extracted 2D facial and 3D head feature parameters. Additionally, five-dimensional heart rate variability (HRV) features were extracted from PPG signals. These ten-dimensional features were fused to construct a fatigue driving dataset. Subsequently, a Long Short-Term Memory (LSTM) network model for fatigue detection was established and optimized using four optimization algorithms: Momentum, Rmsprop, Adam, and SGD. For comparison, Decision Tree (DT), Random Forest (RF), and Bidirectional LSTM (BiLSTM) models were also evaluated. Within the dataset, 2880 samples were designated as the training set, while 720 samples served as the test set. Results: Adam's optimized LSTM fatigue detection model is the most effective, with a model accuracy of 97.36 %, precision of 97.4 %, recall of 97.4 %, and F1 of 0.97. It shows that the model can provide a more timely and accurate prediction and warning for drivers who are already fatigued. |
| format | Article |
| id | doaj-art-1aae97e26d3a42dc89830a302b396c66 |
| institution | Kabale University |
| issn | 2405-8440 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Heliyon |
| spelling | doaj-art-1aae97e26d3a42dc89830a302b396c662024-11-15T06:12:50ZengElsevierHeliyon2405-84402024-11-011021e39479Driver fatigue detection using PPG signal, facial features, head postures with an LSTM modelLu Yu0Xinyi Yang1Hengjian Wei2Jianguo Liu3Bo Li4School of Traffic and Transportation Engineering, Dalian Jiaotong University, Liaoning, Dalian, 116028, China; Corresponding author.School of Traffic and Transportation Engineering, Dalian Jiaotong University, Liaoning, Dalian, 116028, ChinaSchool of Traffic and Transportation Engineering, Dalian Jiaotong University, Liaoning, Dalian, 116028, ChinaSchool of Civil and Resource Engineering from University of Science and Technology Beijing, Beijing, 100083, ChinaSchool of Traffic and Transportation Engineering, Dalian Jiaotong University, Liaoning, Dalian, 116028, ChinaBackground and objective: Background and objective: Human fatigue is a major cause of road traffic accidents. Currently widely used fatigue driving detection methods are based on eyelid closure, vehicle information or physiological parameter detection. However, the detection of each single feature has certain limitations. Which in turn affects the accuracy of detection and the possibility and efficiency of prediction. Methods: This paper introduces a novel driver fatigue detection framework that leverages facial features, head pose, and PPG signals to establish a fatigue detection model. To validate this approach, a real-road driving experiment was conducted, resulting in the acquisition of multi-source feature signal data from 30 drivers. Utilizing a method for locating 68 facial landmarks, we extracted 2D facial and 3D head feature parameters. Additionally, five-dimensional heart rate variability (HRV) features were extracted from PPG signals. These ten-dimensional features were fused to construct a fatigue driving dataset. Subsequently, a Long Short-Term Memory (LSTM) network model for fatigue detection was established and optimized using four optimization algorithms: Momentum, Rmsprop, Adam, and SGD. For comparison, Decision Tree (DT), Random Forest (RF), and Bidirectional LSTM (BiLSTM) models were also evaluated. Within the dataset, 2880 samples were designated as the training set, while 720 samples served as the test set. Results: Adam's optimized LSTM fatigue detection model is the most effective, with a model accuracy of 97.36 %, precision of 97.4 %, recall of 97.4 %, and F1 of 0.97. It shows that the model can provide a more timely and accurate prediction and warning for drivers who are already fatigued.http://www.sciencedirect.com/science/article/pii/S2405844024155107Driver fatiguePPG signalsLSTM modelFacial featureHead posture |
| spellingShingle | Lu Yu Xinyi Yang Hengjian Wei Jianguo Liu Bo Li Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model Heliyon Driver fatigue PPG signals LSTM model Facial feature Head posture |
| title | Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model |
| title_full | Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model |
| title_fullStr | Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model |
| title_full_unstemmed | Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model |
| title_short | Driver fatigue detection using PPG signal, facial features, head postures with an LSTM model |
| title_sort | driver fatigue detection using ppg signal facial features head postures with an lstm model |
| topic | Driver fatigue PPG signals LSTM model Facial feature Head posture |
| url | http://www.sciencedirect.com/science/article/pii/S2405844024155107 |
| work_keys_str_mv | AT luyu driverfatiguedetectionusingppgsignalfacialfeaturesheadpostureswithanlstmmodel AT xinyiyang driverfatiguedetectionusingppgsignalfacialfeaturesheadpostureswithanlstmmodel AT hengjianwei driverfatiguedetectionusingppgsignalfacialfeaturesheadpostureswithanlstmmodel AT jianguoliu driverfatiguedetectionusingppgsignalfacialfeaturesheadpostureswithanlstmmodel AT boli driverfatiguedetectionusingppgsignalfacialfeaturesheadpostureswithanlstmmodel |