Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison
Abstract Purpose To conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-10-01
|
| Series: | Cancer Imaging |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40644-024-00790-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850201240494931968 |
|---|---|
| author | Weibin Zhang Qihui Guo Yuli Zhu Meng Wang Tong Zhang Guangwen Cheng Qi Zhang Hong Ding |
| author_facet | Weibin Zhang Qihui Guo Yuli Zhu Meng Wang Tong Zhang Guangwen Cheng Qi Zhang Hong Ding |
| author_sort | Weibin Zhang |
| collection | DOAJ |
| description | Abstract Purpose To conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation. Methods This retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC). Results Thirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05). Conclusion Cross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures. |
| format | Article |
| id | doaj-art-60707a32a88b4fa6ac2fe84f60aae388 |
| institution | OA Journals |
| issn | 1470-7330 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | BMC |
| record_format | Article |
| series | Cancer Imaging |
| spelling | doaj-art-60707a32a88b4fa6ac2fe84f60aae3882025-08-20T02:12:03ZengBMCCancer Imaging1470-73302024-10-0124111010.1186/s40644-024-00790-9Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparisonWeibin Zhang0Qihui Guo1Yuli Zhu2Meng Wang3Tong Zhang4Guangwen Cheng5Qi Zhang6Hong Ding7Department of Ultrasound, Huashan Hospital, Fudan UniversityThe SMART (Smart Medicine and AI-based Radiology Technology) Lab, School of Communication and Information Engineering, Shanghai UniversityDepartment of Ultrasound, Institute of Ultrasound in Medicine and Engineering, Zhongshan Hospital, Fudan UniversityThe SMART (Smart Medicine and AI-based Radiology Technology) Lab, School of Communication and Information Engineering, Shanghai UniversityDepartment of Ultrasound, Huashan Hospital, Fudan UniversityDepartment of Ultrasound, Huashan Hospital, Fudan UniversityThe SMART (Smart Medicine and AI-based Radiology Technology) Lab, School of Communication and Information Engineering, Shanghai UniversityDepartment of Ultrasound, Huashan Hospital, Fudan UniversityAbstract Purpose To conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation. Methods This retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC). Results Thirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05). Conclusion Cross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.https://doi.org/10.1186/s40644-024-00790-9Deep learningRadiomicsContrast-enhanced ultrasoundMicrovascular invasionHead-to-head comparison |
| spellingShingle | Weibin Zhang Qihui Guo Yuli Zhu Meng Wang Tong Zhang Guangwen Cheng Qi Zhang Hong Ding Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison Cancer Imaging Deep learning Radiomics Contrast-enhanced ultrasound Microvascular invasion Head-to-head comparison |
| title | Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison |
| title_full | Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison |
| title_fullStr | Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison |
| title_full_unstemmed | Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison |
| title_short | Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison |
| title_sort | cross institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma validity robustness and ultrasound modality efficacy comparison |
| topic | Deep learning Radiomics Contrast-enhanced ultrasound Microvascular invasion Head-to-head comparison |
| url | https://doi.org/10.1186/s40644-024-00790-9 |
| work_keys_str_mv | AT weibinzhang crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT qihuiguo crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT yulizhu crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT mengwang crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT tongzhang crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT guangwencheng crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT qizhang crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison AT hongding crossinstitutionalevaluationofdeeplearningandradiomicsmodelsinpredictingmicrovascularinvasioninhepatocellularcarcinomavalidityrobustnessandultrasoundmodalityefficacycomparison |