Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants.
<h4>Background</h4>Soil-transmitted helminth (STH) and Schistosoma mansoni (S. mansoni) infections remain significant public health concerns in tropical and subtropical regions. Deep Convolutional Neural Networks (DCNNs) have already shown promising accuracy in identifying STH and S. man...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-07-01
|
| Series: | PLoS Neglected Tropical Diseases |
| Online Access: | https://doi.org/10.1371/journal.pntd.0013234 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849744628595556352 |
|---|---|
| author | Mohammed Aliy Mohammed Esla Timothy Anzaku Peter Kenneth Ward Bruno Levecke Janarthanan Krishnamoorthy Wesley De Neve Sofie Van Hoecke |
| author_facet | Mohammed Aliy Mohammed Esla Timothy Anzaku Peter Kenneth Ward Bruno Levecke Janarthanan Krishnamoorthy Wesley De Neve Sofie Van Hoecke |
| author_sort | Mohammed Aliy Mohammed |
| collection | DOAJ |
| description | <h4>Background</h4>Soil-transmitted helminth (STH) and Schistosoma mansoni (S. mansoni) infections remain significant public health concerns in tropical and subtropical regions. Deep Convolutional Neural Networks (DCNNs) have already shown promising accuracy in identifying STH and S. mansoni eggs in the same, in-distribution (ID) settings. However, their performance in real-world, out-of-distribution (OOD) scenarios, characterized by variations in image capture devices and the appearance of previously unseen egg types, has not been thoroughly investigated. Assessing the robustness of DCNNs under these challenging conditions is crucial for ensuring their reliability in field diagnostics.<h4>Methodology</h4>Our study addresses the gap in evaluating DCNNs for identifying STH and S. mansoni eggs by rigorously testing multiple variants of the You Only Look Once (YOLO) version 7 model under two OOD conditions: (i) a dataset shift due to a change in the image capture device, and (ii) a combination of this device change and the presence of two egg types not occurring during training. We adopted a 2 [Formula: see text] 3 montage data augmentation strategy to enhance OOD generalization. Additionally, we used the Toolkit for Identifying object Detection Errors (TIDE) and Gradient-weighted Class Activation Mapping (Grad-CAM) to perform a comprehensive analysis of the results.<h4>Principal findings</h4>In ID settings, YOLOv7-E6E outperformed other models, achieving an F1-score of 97.47%. For the OOD scenario involving only a change in the image capture device, the 2 [Formula: see text] 3 montage strategy significantly enhanced performance, increasing precision by 8%, recall by 14.85%, and mAP@IoU0.5 by 21.36%. However, for the more complex OOD scenario that involves both a change in the capture device and the introduction of two previously unseen egg types, the proposed augmentation technique, while beneficial, did not fully address the generalization challenges across all YOLOv7 variants, highlighting the necessity of testing beyond ID scenarios, on which state-of-the-art models predominantly focus.<h4>Conclusions/significance</h4>This study underscores the critical importance of utilizing comprehensive test sets and conducting rigorous OOD evaluations when designing machine learning solutions for STH, S. mansoni or any other helminth infections. Understanding the true capabilities of DCNNs in real-world settings depends on such thorough testing. Expanding AI-driven diagnostic assessments to account for the complexities encountered in the field is essential for creating robust tools that can significantly contribute to the global elimination of STH and S. mansoni infections as public health problems by 2030, a goal put forth by the World Health Organization. |
| format | Article |
| id | doaj-art-7791f0d827cb472ba37ecd35bc549db4 |
| institution | DOAJ |
| issn | 1935-2727 1935-2735 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS Neglected Tropical Diseases |
| spelling | doaj-art-7791f0d827cb472ba37ecd35bc549db42025-08-20T03:13:11ZengPublic Library of Science (PLoS)PLoS Neglected Tropical Diseases1935-27271935-27352025-07-01197e001323410.1371/journal.pntd.0013234Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants.Mohammed Aliy MohammedEsla Timothy AnzakuPeter Kenneth WardBruno LeveckeJanarthanan KrishnamoorthyWesley De NeveSofie Van Hoecke<h4>Background</h4>Soil-transmitted helminth (STH) and Schistosoma mansoni (S. mansoni) infections remain significant public health concerns in tropical and subtropical regions. Deep Convolutional Neural Networks (DCNNs) have already shown promising accuracy in identifying STH and S. mansoni eggs in the same, in-distribution (ID) settings. However, their performance in real-world, out-of-distribution (OOD) scenarios, characterized by variations in image capture devices and the appearance of previously unseen egg types, has not been thoroughly investigated. Assessing the robustness of DCNNs under these challenging conditions is crucial for ensuring their reliability in field diagnostics.<h4>Methodology</h4>Our study addresses the gap in evaluating DCNNs for identifying STH and S. mansoni eggs by rigorously testing multiple variants of the You Only Look Once (YOLO) version 7 model under two OOD conditions: (i) a dataset shift due to a change in the image capture device, and (ii) a combination of this device change and the presence of two egg types not occurring during training. We adopted a 2 [Formula: see text] 3 montage data augmentation strategy to enhance OOD generalization. Additionally, we used the Toolkit for Identifying object Detection Errors (TIDE) and Gradient-weighted Class Activation Mapping (Grad-CAM) to perform a comprehensive analysis of the results.<h4>Principal findings</h4>In ID settings, YOLOv7-E6E outperformed other models, achieving an F1-score of 97.47%. For the OOD scenario involving only a change in the image capture device, the 2 [Formula: see text] 3 montage strategy significantly enhanced performance, increasing precision by 8%, recall by 14.85%, and mAP@IoU0.5 by 21.36%. However, for the more complex OOD scenario that involves both a change in the capture device and the introduction of two previously unseen egg types, the proposed augmentation technique, while beneficial, did not fully address the generalization challenges across all YOLOv7 variants, highlighting the necessity of testing beyond ID scenarios, on which state-of-the-art models predominantly focus.<h4>Conclusions/significance</h4>This study underscores the critical importance of utilizing comprehensive test sets and conducting rigorous OOD evaluations when designing machine learning solutions for STH, S. mansoni or any other helminth infections. Understanding the true capabilities of DCNNs in real-world settings depends on such thorough testing. Expanding AI-driven diagnostic assessments to account for the complexities encountered in the field is essential for creating robust tools that can significantly contribute to the global elimination of STH and S. mansoni infections as public health problems by 2030, a goal put forth by the World Health Organization.https://doi.org/10.1371/journal.pntd.0013234 |
| spellingShingle | Mohammed Aliy Mohammed Esla Timothy Anzaku Peter Kenneth Ward Bruno Levecke Janarthanan Krishnamoorthy Wesley De Neve Sofie Van Hoecke Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants. PLoS Neglected Tropical Diseases |
| title | Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants. |
| title_full | Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants. |
| title_fullStr | Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants. |
| title_full_unstemmed | Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants. |
| title_short | Detecting soil-transmitted helminth and Schistosoma mansoni eggs in Kato-Katz stool smear microscopy images: A comprehensive in- and out-of-distribution evaluation of YOLOv7 variants. |
| title_sort | detecting soil transmitted helminth and schistosoma mansoni eggs in kato katz stool smear microscopy images a comprehensive in and out of distribution evaluation of yolov7 variants |
| url | https://doi.org/10.1371/journal.pntd.0013234 |
| work_keys_str_mv | AT mohammedaliymohammed detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants AT eslatimothyanzaku detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants AT peterkennethward detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants AT brunolevecke detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants AT janarthanankrishnamoorthy detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants AT wesleydeneve detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants AT sofievanhoecke detectingsoiltransmittedhelminthandschistosomamansonieggsinkatokatzstoolsmearmicroscopyimagesacomprehensiveinandoutofdistributionevaluationofyolov7variants |