Deep learning empowered sensor fusion boosts infant movement classification
Abstract Background To assess the integrity of the developing nervous system, the Prechtl general movement assessment (GMA) is recognized for its clinical value in diagnosing neurological impairments in early infancy. GMA has been increasingly augmented through machine learning approaches intending...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Communications Medicine |
| Online Access: | https://doi.org/10.1038/s43856-024-00701-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832594482165448704 |
|---|---|
| author | Tomas Kulvicius Dajie Zhang Luise Poustka Sven Bölte Lennart Jahn Sarah Flügge Marc Kraft Markus Zweckstetter Karin Nielsen-Saines Florentin Wörgötter Peter B. Marschik |
| author_facet | Tomas Kulvicius Dajie Zhang Luise Poustka Sven Bölte Lennart Jahn Sarah Flügge Marc Kraft Markus Zweckstetter Karin Nielsen-Saines Florentin Wörgötter Peter B. Marschik |
| author_sort | Tomas Kulvicius |
| collection | DOAJ |
| description | Abstract Background To assess the integrity of the developing nervous system, the Prechtl general movement assessment (GMA) is recognized for its clinical value in diagnosing neurological impairments in early infancy. GMA has been increasingly augmented through machine learning approaches intending to scale-up its application, circumvent costs in the training of human assessors and further standardize classification of spontaneous motor patterns. Available deep learning tools, all of which are based on single sensor modalities, are however still considerably inferior to that of well-trained human assessors. These approaches are hardly comparable as all models are designed, trained and evaluated on proprietary/silo-data sets. Methods With this study we propose a sensor fusion approach for assessing fidgety movements (FMs). FMs were recorded from 51 typically developing participants. We compared three different sensor modalities (pressure, inertial, and visual sensors). Various combinations and two sensor fusion approaches (late and early fusion) for infant movement classification were tested to evaluate whether a multi-sensor system outperforms single modality assessments. Convolutional neural network (CNN) architectures were used to classify movement patterns. Results The performance of the three-sensor fusion (classification accuracy of 94.5%) is significantly higher than that of any single modality evaluated. Conclusions We show that the sensor fusion approach is a promising avenue for automated classification of infant motor patterns. The development of a robust sensor fusion system may significantly enhance AI-based early recognition of neurofunctions, ultimately facilitating automated early detection of neurodevelopmental conditions. |
| format | Article |
| id | doaj-art-62577f50cb5b4f38aef84e60fc8a5de2 |
| institution | Kabale University |
| issn | 2730-664X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Medicine |
| spelling | doaj-art-62577f50cb5b4f38aef84e60fc8a5de22025-01-19T12:36:54ZengNature PortfolioCommunications Medicine2730-664X2025-01-015111210.1038/s43856-024-00701-wDeep learning empowered sensor fusion boosts infant movement classificationTomas Kulvicius0Dajie Zhang1Luise Poustka2Sven Bölte3Lennart Jahn4Sarah Flügge5Marc Kraft6Markus Zweckstetter7Karin Nielsen-Saines8Florentin Wörgötter9Peter B. Marschik10Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Leibniz ScienceCampus Primate Cognition and German Center for Child and Adolescent Health (DZKJ)Department of Child and Adolescent Psychiatry, University Hospital Heidelberg, Heidelberg UniversityDepartment of Child and Adolescent Psychiatry, University Hospital Heidelberg, Heidelberg UniversityCenter of Neurodevelopmental Disorders (KIND), Department of Women’s and Children’s Health, Center for Psychiatry Research, Karolinska Institutet & Region StockholmChild and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Leibniz ScienceCampus Primate Cognition and German Center for Child and Adolescent Health (DZKJ)Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Leibniz ScienceCampus Primate Cognition and German Center for Child and Adolescent Health (DZKJ)Department of Medical Engineering, Technical University BerlinDepartment of Child and Adolescent Psychiatry, University Hospital Heidelberg, Heidelberg UniversityDepartment of Pediatrics, David Geffen UCLA School of MedicineDepartment for Computational Neuroscience, Third Institute of Physics - Biophysics, Georg-August-University of GöttingenChild and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Leibniz ScienceCampus Primate Cognition and German Center for Child and Adolescent Health (DZKJ)Abstract Background To assess the integrity of the developing nervous system, the Prechtl general movement assessment (GMA) is recognized for its clinical value in diagnosing neurological impairments in early infancy. GMA has been increasingly augmented through machine learning approaches intending to scale-up its application, circumvent costs in the training of human assessors and further standardize classification of spontaneous motor patterns. Available deep learning tools, all of which are based on single sensor modalities, are however still considerably inferior to that of well-trained human assessors. These approaches are hardly comparable as all models are designed, trained and evaluated on proprietary/silo-data sets. Methods With this study we propose a sensor fusion approach for assessing fidgety movements (FMs). FMs were recorded from 51 typically developing participants. We compared three different sensor modalities (pressure, inertial, and visual sensors). Various combinations and two sensor fusion approaches (late and early fusion) for infant movement classification were tested to evaluate whether a multi-sensor system outperforms single modality assessments. Convolutional neural network (CNN) architectures were used to classify movement patterns. Results The performance of the three-sensor fusion (classification accuracy of 94.5%) is significantly higher than that of any single modality evaluated. Conclusions We show that the sensor fusion approach is a promising avenue for automated classification of infant motor patterns. The development of a robust sensor fusion system may significantly enhance AI-based early recognition of neurofunctions, ultimately facilitating automated early detection of neurodevelopmental conditions.https://doi.org/10.1038/s43856-024-00701-w |
| spellingShingle | Tomas Kulvicius Dajie Zhang Luise Poustka Sven Bölte Lennart Jahn Sarah Flügge Marc Kraft Markus Zweckstetter Karin Nielsen-Saines Florentin Wörgötter Peter B. Marschik Deep learning empowered sensor fusion boosts infant movement classification Communications Medicine |
| title | Deep learning empowered sensor fusion boosts infant movement classification |
| title_full | Deep learning empowered sensor fusion boosts infant movement classification |
| title_fullStr | Deep learning empowered sensor fusion boosts infant movement classification |
| title_full_unstemmed | Deep learning empowered sensor fusion boosts infant movement classification |
| title_short | Deep learning empowered sensor fusion boosts infant movement classification |
| title_sort | deep learning empowered sensor fusion boosts infant movement classification |
| url | https://doi.org/10.1038/s43856-024-00701-w |
| work_keys_str_mv | AT tomaskulvicius deeplearningempoweredsensorfusionboostsinfantmovementclassification AT dajiezhang deeplearningempoweredsensorfusionboostsinfantmovementclassification AT luisepoustka deeplearningempoweredsensorfusionboostsinfantmovementclassification AT svenbolte deeplearningempoweredsensorfusionboostsinfantmovementclassification AT lennartjahn deeplearningempoweredsensorfusionboostsinfantmovementclassification AT sarahflugge deeplearningempoweredsensorfusionboostsinfantmovementclassification AT marckraft deeplearningempoweredsensorfusionboostsinfantmovementclassification AT markuszweckstetter deeplearningempoweredsensorfusionboostsinfantmovementclassification AT karinnielsensaines deeplearningempoweredsensorfusionboostsinfantmovementclassification AT florentinworgotter deeplearningempoweredsensorfusionboostsinfantmovementclassification AT peterbmarschik deeplearningempoweredsensorfusionboostsinfantmovementclassification |