Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose
In recent years, significant research has been conducted on video-based human pose estimation (HPE). While monocular two-dimensional (2D) HPE has been shown to achieve high performance, monocular three-dimensional (3D) HPE poses a more challenging problem. However, since human motion happens in a 3D...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/23/7772 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850106522220101632 |
|---|---|
| author | Sebastian Dill Arjang Ahmadi Martin Grimmer Dennis Haufe Maurice Rohr Yanhua Zhao Maziar Sharbafi Christoph Hoog Antink |
| author_facet | Sebastian Dill Arjang Ahmadi Martin Grimmer Dennis Haufe Maurice Rohr Yanhua Zhao Maziar Sharbafi Christoph Hoog Antink |
| author_sort | Sebastian Dill |
| collection | DOAJ |
| description | In recent years, significant research has been conducted on video-based human pose estimation (HPE). While monocular two-dimensional (2D) HPE has been shown to achieve high performance, monocular three-dimensional (3D) HPE poses a more challenging problem. However, since human motion happens in a 3D space, 3D HPE offers a more accurate representation of the human, granting increased usability for complex tasks like analysis of physical exercise. We propose a method based on MediaPipe Pose, 2D HPE on stereo cameras and a fusion algorithm without prior stereo calibration to reconstruct 3D poses, combining the advantages of high accuracy in 2D HPE with the increased usability of 3D coordinates. We evaluate this method on a self-recorded database focused on physical exercise to research what accuracy can be achieved and whether this accuracy is sufficient to recognize errors in exercise performance. We find that our method achieves significantly improved performance compared to monocular 3D HPE (median RMSE of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>30.1</mn></mrow></semantics></math></inline-formula> compared to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>56.3</mn></mrow></semantics></math></inline-formula>, <i>p</i>-value below <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mn>10</mn><mrow><mo>−</mo><mn>6</mn></mrow></msup></semantics></math></inline-formula>) and can show that the performance is sufficient for error recognition. |
| format | Article |
| id | doaj-art-ca97c48fe6cb4ca5be611bb639058a83 |
| institution | OA Journals |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-ca97c48fe6cb4ca5be611bb639058a832025-08-20T02:38:48ZengMDPI AGSensors1424-82202024-12-012423777210.3390/s24237772Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe PoseSebastian Dill0Arjang Ahmadi1Martin Grimmer2Dennis Haufe3Maurice Rohr4Yanhua Zhao5Maziar Sharbafi6Christoph Hoog Antink7KIS*MED (AI Systems in Medicine), Technische Universität Darmstadt, 64283 Darmstadt, GermanyLauflabor (Locomotion Laboratory), Centre for Cognitive Science, Technische Universität Darmstadt, 64289 Darmstadt, GermanyLauflabor (Locomotion Laboratory), Centre for Cognitive Science, Technische Universität Darmstadt, 64289 Darmstadt, GermanyLauflabor (Locomotion Laboratory), Centre for Cognitive Science, Technische Universität Darmstadt, 64289 Darmstadt, GermanyKIS*MED (AI Systems in Medicine), Technische Universität Darmstadt, 64283 Darmstadt, GermanyKIS*MED (AI Systems in Medicine), Technische Universität Darmstadt, 64283 Darmstadt, GermanyLauflabor (Locomotion Laboratory), Centre for Cognitive Science, Technische Universität Darmstadt, 64289 Darmstadt, GermanyKIS*MED (AI Systems in Medicine), Technische Universität Darmstadt, 64283 Darmstadt, GermanyIn recent years, significant research has been conducted on video-based human pose estimation (HPE). While monocular two-dimensional (2D) HPE has been shown to achieve high performance, monocular three-dimensional (3D) HPE poses a more challenging problem. However, since human motion happens in a 3D space, 3D HPE offers a more accurate representation of the human, granting increased usability for complex tasks like analysis of physical exercise. We propose a method based on MediaPipe Pose, 2D HPE on stereo cameras and a fusion algorithm without prior stereo calibration to reconstruct 3D poses, combining the advantages of high accuracy in 2D HPE with the increased usability of 3D coordinates. We evaluate this method on a self-recorded database focused on physical exercise to research what accuracy can be achieved and whether this accuracy is sufficient to recognize errors in exercise performance. We find that our method achieves significantly improved performance compared to monocular 3D HPE (median RMSE of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>30.1</mn></mrow></semantics></math></inline-formula> compared to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>56.3</mn></mrow></semantics></math></inline-formula>, <i>p</i>-value below <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mn>10</mn><mrow><mo>−</mo><mn>6</mn></mrow></msup></semantics></math></inline-formula>) and can show that the performance is sufficient for error recognition.https://www.mdpi.com/1424-8220/24/23/7772computer visionhuman pose estimationinformation fusionMediaPipe Pose |
| spellingShingle | Sebastian Dill Arjang Ahmadi Martin Grimmer Dennis Haufe Maurice Rohr Yanhua Zhao Maziar Sharbafi Christoph Hoog Antink Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose Sensors computer vision human pose estimation information fusion MediaPipe Pose |
| title | Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose |
| title_full | Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose |
| title_fullStr | Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose |
| title_full_unstemmed | Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose |
| title_short | Accuracy Evaluation of 3D Pose Reconstruction Algorithms Through Stereo Camera Information Fusion for Physical Exercises with MediaPipe Pose |
| title_sort | accuracy evaluation of 3d pose reconstruction algorithms through stereo camera information fusion for physical exercises with mediapipe pose |
| topic | computer vision human pose estimation information fusion MediaPipe Pose |
| url | https://www.mdpi.com/1424-8220/24/23/7772 |
| work_keys_str_mv | AT sebastiandill accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT arjangahmadi accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT martingrimmer accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT dennishaufe accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT mauricerohr accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT yanhuazhao accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT maziarsharbafi accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose AT christophhoogantink accuracyevaluationof3dposereconstructionalgorithmsthroughstereocamerainformationfusionforphysicalexerciseswithmediapipepose |