Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy
Early detection of Cerebral Palsy (CP) is crucial for effective intervention and monitoring. This paper tests the reliability and applicability of Explainable AI (XAI) methods using a deep learning method that predicts CP by analyzing skeletal data extracted from video recordings of infant movements...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10820328/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832592927266701312 |
|---|---|
| author | Kimji N. Pellano Inga Strumke Daniel Groos Lars Adde Espen F. Alexander Ihlen |
| author_facet | Kimji N. Pellano Inga Strumke Daniel Groos Lars Adde Espen F. Alexander Ihlen |
| author_sort | Kimji N. Pellano |
| collection | DOAJ |
| description | Early detection of Cerebral Palsy (CP) is crucial for effective intervention and monitoring. This paper tests the reliability and applicability of Explainable AI (XAI) methods using a deep learning method that predicts CP by analyzing skeletal data extracted from video recordings of infant movements. Specifically, we use XAI evaluation metrics — namely faithfulness and stability — to quantitatively assess the reliability of Class Activation Mapping (CAM) and Gradient-weighted Class Activation Mapping (Grad-CAM) in this specific medical application. We utilize a unique dataset of infant movements and apply skeleton data perturbations without distorting the original dynamics of the infant movements. Our CP prediction model utilizes an ensemble approach, so we evaluate the XAI metrics performances for both the overall ensemble and the individual models. Our findings indicate that both XAI methods effectively identify key body points influencing CP predictions and that the explanations are robust against minor data perturbations. Grad-CAM significantly outperforms CAM in the Relative Input Stability velocity (RISv) metric, which measures stability in terms of velocity. In contrast, CAM performs better in the Relative Input Stability bone (RISb) metric, which relates to bone stability, and the Relative Representation Stability (RRS) metric, which assesses internal representation robustness. Individual models within the ensemble show varied results, and neither CAM nor Grad-CAM consistently outperform the other, with the ensemble approach providing a representation of outcomes from its constituent models. Both CAM and Grad-CAM also perform significantly better than random attribution, supporting the robustness of these XAI methods. Our work demonstrates that XAI methods can offer reliable and stable explanations for CP prediction models. Future studies should further investigate how the explanations can enhance our understanding of specific movement patterns characterizing healthy and pathological development. |
| format | Article |
| id | doaj-art-62279fe9ca6b47999e7d673106206f3a |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-62279fe9ca6b47999e7d673106206f3a2025-01-21T00:02:00ZengIEEEIEEE Access2169-35362025-01-0113101261013810.1109/ACCESS.2025.352557110820328Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral PalsyKimji N. Pellano0https://orcid.org/0000-0002-5423-9418Inga Strumke1https://orcid.org/0000-0003-1820-6544Daniel Groos2https://orcid.org/0000-0003-0676-2324Lars Adde3https://orcid.org/0000-0001-5532-0034Espen F. Alexander Ihlen4https://orcid.org/0000-0002-2469-1809Department of Neuromedicine and Movement Science, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Trondheim, NorwayDepartment of Computer Science, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Trondheim, NorwayDepartment of Neuromedicine and Movement Science, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Trondheim, NorwayDepartment of Clinical and Molecular Medicine, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Trondheim, NorwayDepartment of Neuromedicine and Movement Science, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Trondheim, NorwayEarly detection of Cerebral Palsy (CP) is crucial for effective intervention and monitoring. This paper tests the reliability and applicability of Explainable AI (XAI) methods using a deep learning method that predicts CP by analyzing skeletal data extracted from video recordings of infant movements. Specifically, we use XAI evaluation metrics — namely faithfulness and stability — to quantitatively assess the reliability of Class Activation Mapping (CAM) and Gradient-weighted Class Activation Mapping (Grad-CAM) in this specific medical application. We utilize a unique dataset of infant movements and apply skeleton data perturbations without distorting the original dynamics of the infant movements. Our CP prediction model utilizes an ensemble approach, so we evaluate the XAI metrics performances for both the overall ensemble and the individual models. Our findings indicate that both XAI methods effectively identify key body points influencing CP predictions and that the explanations are robust against minor data perturbations. Grad-CAM significantly outperforms CAM in the Relative Input Stability velocity (RISv) metric, which measures stability in terms of velocity. In contrast, CAM performs better in the Relative Input Stability bone (RISb) metric, which relates to bone stability, and the Relative Representation Stability (RRS) metric, which assesses internal representation robustness. Individual models within the ensemble show varied results, and neither CAM nor Grad-CAM consistently outperform the other, with the ensemble approach providing a representation of outcomes from its constituent models. Both CAM and Grad-CAM also perform significantly better than random attribution, supporting the robustness of these XAI methods. Our work demonstrates that XAI methods can offer reliable and stable explanations for CP prediction models. Future studies should further investigate how the explanations can enhance our understanding of specific movement patterns characterizing healthy and pathological development.https://ieeexplore.ieee.org/document/10820328/Explainable AICAMGrad-CAMskeleton dataCerebral Palsy |
| spellingShingle | Kimji N. Pellano Inga Strumke Daniel Groos Lars Adde Espen F. Alexander Ihlen Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy IEEE Access Explainable AI CAM Grad-CAM skeleton data Cerebral Palsy |
| title | Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy |
| title_full | Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy |
| title_fullStr | Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy |
| title_full_unstemmed | Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy |
| title_short | Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy |
| title_sort | evaluating explainable ai methods in deep learning models for early detection of cerebral palsy |
| topic | Explainable AI CAM Grad-CAM skeleton data Cerebral Palsy |
| url | https://ieeexplore.ieee.org/document/10820328/ |
| work_keys_str_mv | AT kimjinpellano evaluatingexplainableaimethodsindeeplearningmodelsforearlydetectionofcerebralpalsy AT ingastrumke evaluatingexplainableaimethodsindeeplearningmodelsforearlydetectionofcerebralpalsy AT danielgroos evaluatingexplainableaimethodsindeeplearningmodelsforearlydetectionofcerebralpalsy AT larsadde evaluatingexplainableaimethodsindeeplearningmodelsforearlydetectionofcerebralpalsy AT espenfalexanderihlen evaluatingexplainableaimethodsindeeplearningmodelsforearlydetectionofcerebralpalsy |