Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis
Background. Retinopathy of prematurity (ROP) occurs in preterm infants and may contribute to blindness. Deep learning (DL) models have been used for ophthalmologic diagnoses. We performed a systematic review and meta-analysis of published evidence to summarize and evaluate the diagnostic accuracy of...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-01-01
|
| Series: | Journal of Ophthalmology |
| Online Access: | http://dx.doi.org/10.1155/2021/8883946 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832555842498461696 |
|---|---|
| author | Jingjing Zhang Yangyang Liu Toshiharu Mitsuhashi Toshihiko Matsuo |
| author_facet | Jingjing Zhang Yangyang Liu Toshiharu Mitsuhashi Toshihiko Matsuo |
| author_sort | Jingjing Zhang |
| collection | DOAJ |
| description | Background. Retinopathy of prematurity (ROP) occurs in preterm infants and may contribute to blindness. Deep learning (DL) models have been used for ophthalmologic diagnoses. We performed a systematic review and meta-analysis of published evidence to summarize and evaluate the diagnostic accuracy of DL algorithms for ROP by fundus images. Methods. We searched PubMed, EMBASE, Web of Science, and Institute of Electrical and Electronics Engineers Xplore Digital Library on June 13, 2021, for studies using a DL algorithm to distinguish individuals with ROP of different grades, which provided accuracy measurements. The pooled sensitivity and specificity values and the area under the curve (AUC) of summary receiver operating characteristics curves (SROC) summarized overall test performance. The performances in validation and test datasets were assessed together and separately. Subgroup analyses were conducted between the definition and grades of ROP. Threshold and nonthreshold effects were tested to assess biases and evaluate accuracy factors associated with DL models. Results. Nine studies with fifteen classifiers were included in our meta-analysis. A total of 521,586 objects were applied to DL models. For combined validation and test datasets in each study, the pooled sensitivity and specificity were 0.953 (95% confidence intervals (CI): 0.946–0.959) and 0.975 (0.973–0.977), respectively, and the AUC was 0.984 (0.978–0.989). For the validation dataset and test dataset, the AUC was 0.977 (0.968–0.986) and 0.987 (0.982–0.992), respectively. In the subgroup analysis of ROP vs. normal and differentiation of two ROP grades, the AUC was 0.990 (0.944–0.994) and 0.982 (0.964–0.999), respectively. Conclusions. Our study shows that DL models can play an essential role in detecting and grading ROP with high sensitivity, specificity, and repeatability. The application of a DL-based automated system may improve ROP screening and diagnosis in the future. |
| format | Article |
| id | doaj-art-12f7f46571bb4cfc944e044b846347d6 |
| institution | Kabale University |
| issn | 2090-004X 2090-0058 |
| language | English |
| publishDate | 2021-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Journal of Ophthalmology |
| spelling | doaj-art-12f7f46571bb4cfc944e044b846347d62025-02-03T05:47:05ZengWileyJournal of Ophthalmology2090-004X2090-00582021-01-01202110.1155/2021/88839468883946Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-AnalysisJingjing Zhang0Yangyang Liu1Toshiharu Mitsuhashi2Toshihiko Matsuo3Department of Regenerative and Reconstructive Medicine (Ophthalmology), Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama 7008530, JapanDepartment of Epidemiology, Graduate School of Medicine, Dentistry and Pharmaceutical Sciences, Okayama University, Okayama 7008558, JapanCenter for Innovative Clinical Medicine, Okayama University Hospital, Okayama University, Okayama 7008558, JapanDepartment of Regenerative and Reconstructive Medicine (Ophthalmology), Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama 7008530, JapanBackground. Retinopathy of prematurity (ROP) occurs in preterm infants and may contribute to blindness. Deep learning (DL) models have been used for ophthalmologic diagnoses. We performed a systematic review and meta-analysis of published evidence to summarize and evaluate the diagnostic accuracy of DL algorithms for ROP by fundus images. Methods. We searched PubMed, EMBASE, Web of Science, and Institute of Electrical and Electronics Engineers Xplore Digital Library on June 13, 2021, for studies using a DL algorithm to distinguish individuals with ROP of different grades, which provided accuracy measurements. The pooled sensitivity and specificity values and the area under the curve (AUC) of summary receiver operating characteristics curves (SROC) summarized overall test performance. The performances in validation and test datasets were assessed together and separately. Subgroup analyses were conducted between the definition and grades of ROP. Threshold and nonthreshold effects were tested to assess biases and evaluate accuracy factors associated with DL models. Results. Nine studies with fifteen classifiers were included in our meta-analysis. A total of 521,586 objects were applied to DL models. For combined validation and test datasets in each study, the pooled sensitivity and specificity were 0.953 (95% confidence intervals (CI): 0.946–0.959) and 0.975 (0.973–0.977), respectively, and the AUC was 0.984 (0.978–0.989). For the validation dataset and test dataset, the AUC was 0.977 (0.968–0.986) and 0.987 (0.982–0.992), respectively. In the subgroup analysis of ROP vs. normal and differentiation of two ROP grades, the AUC was 0.990 (0.944–0.994) and 0.982 (0.964–0.999), respectively. Conclusions. Our study shows that DL models can play an essential role in detecting and grading ROP with high sensitivity, specificity, and repeatability. The application of a DL-based automated system may improve ROP screening and diagnosis in the future.http://dx.doi.org/10.1155/2021/8883946 |
| spellingShingle | Jingjing Zhang Yangyang Liu Toshiharu Mitsuhashi Toshihiko Matsuo Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis Journal of Ophthalmology |
| title | Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis |
| title_full | Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis |
| title_fullStr | Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis |
| title_full_unstemmed | Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis |
| title_short | Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis |
| title_sort | accuracy of deep learning algorithms for the diagnosis of retinopathy of prematurity by fundus images a systematic review and meta analysis |
| url | http://dx.doi.org/10.1155/2021/8883946 |
| work_keys_str_mv | AT jingjingzhang accuracyofdeeplearningalgorithmsforthediagnosisofretinopathyofprematuritybyfundusimagesasystematicreviewandmetaanalysis AT yangyangliu accuracyofdeeplearningalgorithmsforthediagnosisofretinopathyofprematuritybyfundusimagesasystematicreviewandmetaanalysis AT toshiharumitsuhashi accuracyofdeeplearningalgorithmsforthediagnosisofretinopathyofprematuritybyfundusimagesasystematicreviewandmetaanalysis AT toshihikomatsuo accuracyofdeeplearningalgorithmsforthediagnosisofretinopathyofprematuritybyfundusimagesasystematicreviewandmetaanalysis |