Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening
Abstract Purpose This study aimed to compare the cost-effectiveness of AI-based approaches with manual approaches in ultrasound image quality control (QC). Methods Eligible ultrasonographers and pregnant volunteers were prospectively recruited from the Hunan Maternal and Child Health Hospital in May...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-12-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-024-06477-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850253831170949120 |
|---|---|
| author | Yihan Tan Yulin Peng Liangyu Guo Dongmei Liu Yingchun Luo |
| author_facet | Yihan Tan Yulin Peng Liangyu Guo Dongmei Liu Yingchun Luo |
| author_sort | Yihan Tan |
| collection | DOAJ |
| description | Abstract Purpose This study aimed to compare the cost-effectiveness of AI-based approaches with manual approaches in ultrasound image quality control (QC). Methods Eligible ultrasonographers and pregnant volunteers were prospectively recruited from the Hunan Maternal and Child Health Hospital in May 2023. The ultrasonographers were randomly and evenly assigned to either the AI or Manual QC groups with baseline scores determined in June-July. From August to October, these groups received real-time AI or post-scan manual QC with post-interventional scores recorded monthly. We applied the repeated measures analysis of variance to analyze the between-subject and within-subject effectiveness and time trends in effectiveness (QC score improvement) assessment. An extra 50 pregnant volunteers underwent real-time manual QC, with their screening images utilized for post-scan AI and manual QC. The time cost of real-time AI QC was zero since it only required trainees’ involvement. We used Friedman’s M and Quade tests to compare multiple independent medians in cost assessment. Results This study recruited 14 ultrasonographers, equally divided into the AI and Manual QC groups. No significant difference existed between the groups concerning age, service year in perinatal diagnosis, male proportion, and QC frequency. The simple effect of the group revealed that the AI QC method outperformed the Manual QC method at least once (F = 13.113, P = 0.004, η 2 = 0.522). The simple effect of the month in the AI QC groups indicated an improvement in the mean QC scores (F = 9.827, P = 0.003, η 2 = 0.747) while that of manual QC groups suggested no improvement (F = 0.144, P = 0.931, η 2 = 0.041). Baseline scores were equal in June-July (F = 0.031, P = 0.864, η 2 = 0.003). However, the AI QC group surpassed the Manual QC group in August (F = 14.579, P = 0.002, η 2 = 0.549), September (F = 28.590, P < 0.001, η 2 = 0.704), and October (F = 35.411, P < 0.001, η 2 = 0.747). Within the Manual QC group, no significant differences were found in scores between June-July and August, September, or October (all P values of 1.000, nominal significance level of 0.0083). In contrast, the AI QC group showed significantly higher scores in August, September, and October compared to June-July (all P values of 0.001, nominal significance level of 0.0083). The time costs of real-time AI QC, post-scan AI QC, post-scan manual QC, and real-time manual QC were 0 s, 13.76 s (interquartile range, IQR: 4.79–46.79 s), 1239.50 s (IQR: 1141.00–1311.25 s), and 1541.00 s (IQR: 1453.50–1635.25 s), with significant differences in both overall and multiple comparisons. Conclusions The AI QC method, more cost-effective than the manual method, shows great potential for application in image QC scenarios. The AI QC can enhance operators’ skills in perinatal ultrasound screening, while the manual method can only maintain the existing level. |
| format | Article |
| id | doaj-art-6c52b60c0fd74cecb1925cf8b1859444 |
| institution | OA Journals |
| issn | 1472-6920 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj-art-6c52b60c0fd74cecb1925cf8b18594442025-08-20T01:57:16ZengBMCBMC Medical Education1472-69202024-12-0124111110.1186/s12909-024-06477-wCost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screeningYihan Tan0Yulin Peng1Liangyu Guo2Dongmei Liu3Yingchun Luo4Department of Ultrasonography, Hunan Provincial Maternal and Child Health Care HospitalDepartment of Ultrasonography, Hunan Provincial Maternal and Child Health Care HospitalDepartment of Ultrasonography, Hunan Provincial Maternal and Child Health Care HospitalDepartment of Ultrasonography, Hunan Provincial Maternal and Child Health Care HospitalDepartment of Ultrasonography, Hunan Provincial Maternal and Child Health Care HospitalAbstract Purpose This study aimed to compare the cost-effectiveness of AI-based approaches with manual approaches in ultrasound image quality control (QC). Methods Eligible ultrasonographers and pregnant volunteers were prospectively recruited from the Hunan Maternal and Child Health Hospital in May 2023. The ultrasonographers were randomly and evenly assigned to either the AI or Manual QC groups with baseline scores determined in June-July. From August to October, these groups received real-time AI or post-scan manual QC with post-interventional scores recorded monthly. We applied the repeated measures analysis of variance to analyze the between-subject and within-subject effectiveness and time trends in effectiveness (QC score improvement) assessment. An extra 50 pregnant volunteers underwent real-time manual QC, with their screening images utilized for post-scan AI and manual QC. The time cost of real-time AI QC was zero since it only required trainees’ involvement. We used Friedman’s M and Quade tests to compare multiple independent medians in cost assessment. Results This study recruited 14 ultrasonographers, equally divided into the AI and Manual QC groups. No significant difference existed between the groups concerning age, service year in perinatal diagnosis, male proportion, and QC frequency. The simple effect of the group revealed that the AI QC method outperformed the Manual QC method at least once (F = 13.113, P = 0.004, η 2 = 0.522). The simple effect of the month in the AI QC groups indicated an improvement in the mean QC scores (F = 9.827, P = 0.003, η 2 = 0.747) while that of manual QC groups suggested no improvement (F = 0.144, P = 0.931, η 2 = 0.041). Baseline scores were equal in June-July (F = 0.031, P = 0.864, η 2 = 0.003). However, the AI QC group surpassed the Manual QC group in August (F = 14.579, P = 0.002, η 2 = 0.549), September (F = 28.590, P < 0.001, η 2 = 0.704), and October (F = 35.411, P < 0.001, η 2 = 0.747). Within the Manual QC group, no significant differences were found in scores between June-July and August, September, or October (all P values of 1.000, nominal significance level of 0.0083). In contrast, the AI QC group showed significantly higher scores in August, September, and October compared to June-July (all P values of 0.001, nominal significance level of 0.0083). The time costs of real-time AI QC, post-scan AI QC, post-scan manual QC, and real-time manual QC were 0 s, 13.76 s (interquartile range, IQR: 4.79–46.79 s), 1239.50 s (IQR: 1141.00–1311.25 s), and 1541.00 s (IQR: 1453.50–1635.25 s), with significant differences in both overall and multiple comparisons. Conclusions The AI QC method, more cost-effective than the manual method, shows great potential for application in image QC scenarios. The AI QC can enhance operators’ skills in perinatal ultrasound screening, while the manual method can only maintain the existing level.https://doi.org/10.1186/s12909-024-06477-wUltrasonography, prenatalArtificial intelligenceEducation, medical, continuingCost-benefit analysisQuality control |
| spellingShingle | Yihan Tan Yulin Peng Liangyu Guo Dongmei Liu Yingchun Luo Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening BMC Medical Education Ultrasonography, prenatal Artificial intelligence Education, medical, continuing Cost-benefit analysis Quality control |
| title | Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening |
| title_full | Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening |
| title_fullStr | Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening |
| title_full_unstemmed | Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening |
| title_short | Cost-effectiveness analysis of AI-based image quality control for perinatal ultrasound screening |
| title_sort | cost effectiveness analysis of ai based image quality control for perinatal ultrasound screening |
| topic | Ultrasonography, prenatal Artificial intelligence Education, medical, continuing Cost-benefit analysis Quality control |
| url | https://doi.org/10.1186/s12909-024-06477-w |
| work_keys_str_mv | AT yihantan costeffectivenessanalysisofaibasedimagequalitycontrolforperinatalultrasoundscreening AT yulinpeng costeffectivenessanalysisofaibasedimagequalitycontrolforperinatalultrasoundscreening AT liangyuguo costeffectivenessanalysisofaibasedimagequalitycontrolforperinatalultrasoundscreening AT dongmeiliu costeffectivenessanalysisofaibasedimagequalitycontrolforperinatalultrasoundscreening AT yingchunluo costeffectivenessanalysisofaibasedimagequalitycontrolforperinatalultrasoundscreening |