Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images
Aim To evaluate the efficacy of automated machine learning (AutoML) models in detecting fundus diseases using ocular B-scan ultrasound images.Methods Ophthalmologists annotated two B-scan ultrasound image datasets to develop three AutoML models—single-label, multi-class single-label and multi-label—...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2024-12-01
|
| Series: | BMJ Open Ophthalmology |
| Online Access: | https://bmjophth.bmj.com/content/9/1/e001873.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850115177408626688 |
|---|---|
| author | Chen Zhao Rui Jiang Qian Chen Qiaoling Wei |
| author_facet | Chen Zhao Rui Jiang Qian Chen Qiaoling Wei |
| author_sort | Chen Zhao |
| collection | DOAJ |
| description | Aim To evaluate the efficacy of automated machine learning (AutoML) models in detecting fundus diseases using ocular B-scan ultrasound images.Methods Ophthalmologists annotated two B-scan ultrasound image datasets to develop three AutoML models—single-label, multi-class single-label and multi-label—on the Vertex artificial intelligence (AI) platform. Performance of these models was compared among themselves and against existing bespoke models for binary classification tasks.Results The training set involved 3938 images from 1378 patients, while batch predictions used an additional set of 336 images from 180 patients. The single-label AutoML model, trained on normal and abnormal fundus images, achieved an area under the precision-recall curve (AUPRC) of 0.9943. The multi-class single-label model, focused on single-pathology images, recorded an AUPRC of 0.9617, with performance metrics of these two single-label models proving comparable to those of previously published models. The multi-label model, designed to detect both single and multiple pathologies, posted an AUPRC of 0.9650. Pathology classification AUPRCs for the multi-class single-label model ranged from 0.9277 to 1.0000 and from 0.8780 to 0.9980 for the multi-label model. Batch prediction accuracies ranged from 86.57% to 97.65% for various fundus conditions in the multi-label AutoML model. Statistical analysis demonstrated that the single-label model significantly outperformed the other two models in all evaluated metrics (p<0.05).Conclusion AutoML models, developed by clinicians, effectively detected multiple fundus lesions with performance on par with that of deep-learning models crafted by AI specialists. This underscores AutoML’s potential to revolutionise ophthalmologic diagnostics, facilitating broader accessibility and application of sophisticated diagnostic technologies. |
| format | Article |
| id | doaj-art-058d05b8b8f64418a8b7c7f96d975ea7 |
| institution | OA Journals |
| issn | 2397-3269 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open Ophthalmology |
| spelling | doaj-art-058d05b8b8f64418a8b7c7f96d975ea72025-08-20T02:36:39ZengBMJ Publishing GroupBMJ Open Ophthalmology2397-32692024-12-019110.1136/bmjophth-2024-001873Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound imagesChen Zhao0Rui Jiang1Qian Chen2Qiaoling Wei3Eye Institute and Department of Ophthalmology, Eye & ENT Hospital, Fudan University, Shanghai, ChinaEye Institute and Department of Ophthalmology, Eye & ENT Hospital, Fudan University, Shanghai, ChinaEye Institute and Department of Ophthalmology, Eye & ENT Hospital, Fudan University, Shanghai, ChinaEye Institute and Department of Ophthalmology, Eye & ENT Hospital, Fudan University, Shanghai, ChinaAim To evaluate the efficacy of automated machine learning (AutoML) models in detecting fundus diseases using ocular B-scan ultrasound images.Methods Ophthalmologists annotated two B-scan ultrasound image datasets to develop three AutoML models—single-label, multi-class single-label and multi-label—on the Vertex artificial intelligence (AI) platform. Performance of these models was compared among themselves and against existing bespoke models for binary classification tasks.Results The training set involved 3938 images from 1378 patients, while batch predictions used an additional set of 336 images from 180 patients. The single-label AutoML model, trained on normal and abnormal fundus images, achieved an area under the precision-recall curve (AUPRC) of 0.9943. The multi-class single-label model, focused on single-pathology images, recorded an AUPRC of 0.9617, with performance metrics of these two single-label models proving comparable to those of previously published models. The multi-label model, designed to detect both single and multiple pathologies, posted an AUPRC of 0.9650. Pathology classification AUPRCs for the multi-class single-label model ranged from 0.9277 to 1.0000 and from 0.8780 to 0.9980 for the multi-label model. Batch prediction accuracies ranged from 86.57% to 97.65% for various fundus conditions in the multi-label AutoML model. Statistical analysis demonstrated that the single-label model significantly outperformed the other two models in all evaluated metrics (p<0.05).Conclusion AutoML models, developed by clinicians, effectively detected multiple fundus lesions with performance on par with that of deep-learning models crafted by AI specialists. This underscores AutoML’s potential to revolutionise ophthalmologic diagnostics, facilitating broader accessibility and application of sophisticated diagnostic technologies.https://bmjophth.bmj.com/content/9/1/e001873.full |
| spellingShingle | Chen Zhao Rui Jiang Qian Chen Qiaoling Wei Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images BMJ Open Ophthalmology |
| title | Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images |
| title_full | Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images |
| title_fullStr | Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images |
| title_full_unstemmed | Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images |
| title_short | Performance of automated machine learning in detecting fundus diseases based on ophthalmologic B-scan ultrasound images |
| title_sort | performance of automated machine learning in detecting fundus diseases based on ophthalmologic b scan ultrasound images |
| url | https://bmjophth.bmj.com/content/9/1/e001873.full |
| work_keys_str_mv | AT chenzhao performanceofautomatedmachinelearningindetectingfundusdiseasesbasedonophthalmologicbscanultrasoundimages AT ruijiang performanceofautomatedmachinelearningindetectingfundusdiseasesbasedonophthalmologicbscanultrasoundimages AT qianchen performanceofautomatedmachinelearningindetectingfundusdiseasesbasedonophthalmologicbscanultrasoundimages AT qiaolingwei performanceofautomatedmachinelearningindetectingfundusdiseasesbasedonophthalmologicbscanultrasoundimages |