A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs
BackgroundVessel segmentation in fundus photography has become a cornerstone technique for disease analysis. Within this field, Ultra-WideField (UWF) fundus images offer distinct advantages, including an expansive imaging range, detailed lesion data, and minimal adverse effects. However, the high re...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Cell and Developmental Biology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcell.2024.1532228/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850150041775243264 |
|---|---|
| author | Qihang Xie Qihang Xie Xuefei Li Yuanyuan Li Yuanyuan Li Jiayi Lu Jiayi Lu Shaodong Ma Yitian Zhao Yitian Zhao Jiong Zhang Jiong Zhang |
| author_facet | Qihang Xie Qihang Xie Xuefei Li Yuanyuan Li Yuanyuan Li Jiayi Lu Jiayi Lu Shaodong Ma Yitian Zhao Yitian Zhao Jiong Zhang Jiong Zhang |
| author_sort | Qihang Xie |
| collection | DOAJ |
| description | BackgroundVessel segmentation in fundus photography has become a cornerstone technique for disease analysis. Within this field, Ultra-WideField (UWF) fundus images offer distinct advantages, including an expansive imaging range, detailed lesion data, and minimal adverse effects. However, the high resolution and low contrast inherent to UWF fundus images present significant challenges for accurate segmentation using deep learning methods, thereby complicating disease analysis in this context.MethodsTo address these issues, this study introduces M3B-Net, a novel multi-modal, multi-branch framework that leverages fundus fluorescence angiography (FFA) images to improve retinal vessel segmentation in UWF fundus images. Specifically, M3B-Net tackles the low segmentation accuracy caused by the inherently low contrast of UWF fundus images. Additionally, we propose an enhanced UWF-based segmentation network in M3B-Net, specifically designed to improve the segmentation of fine retinal vessels. The segmentation network includes the Selective Fusion Module (SFM), which enhances feature extraction within the segmentation network by integrating features generated during the FFA imaging process. To further address the challenges of high-resolution UWF fundus images, we introduce a Local Perception Fusion Module (LPFM) to mitigate context loss during the segmentation cut-patch process. Complementing this, the Attention-Guided Upsampling Module (AUM) enhances segmentation performance through convolution operations guided by attention mechanisms.ResultsExtensive experimental evaluations demonstrate that our approach significantly outperforms existing state-of-the-art methods for UWF fundus image segmentation. |
| format | Article |
| id | doaj-art-e63ceded157249c786f4010f7aa8376f |
| institution | OA Journals |
| issn | 2296-634X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Cell and Developmental Biology |
| spelling | doaj-art-e63ceded157249c786f4010f7aa8376f2025-08-20T02:26:41ZengFrontiers Media S.A.Frontiers in Cell and Developmental Biology2296-634X2025-01-011210.3389/fcell.2024.15322281532228A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographsQihang Xie0Qihang Xie1Xuefei Li2Yuanyuan Li3Yuanyuan Li4Jiayi Lu5Jiayi Lu6Shaodong Ma7Yitian Zhao8Yitian Zhao9Jiong Zhang10Jiong Zhang11Cixi Biomedical Research Institute, Wenzhou Medical University, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaCixi Biomedical Research Institute, Wenzhou Medical University, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaCixi Biomedical Research Institute, Wenzhou Medical University, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaCixi Biomedical Research Institute, Wenzhou Medical University, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaCixi Biomedical Research Institute, Wenzhou Medical University, Ningbo, ChinaLaboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, ChinaBackgroundVessel segmentation in fundus photography has become a cornerstone technique for disease analysis. Within this field, Ultra-WideField (UWF) fundus images offer distinct advantages, including an expansive imaging range, detailed lesion data, and minimal adverse effects. However, the high resolution and low contrast inherent to UWF fundus images present significant challenges for accurate segmentation using deep learning methods, thereby complicating disease analysis in this context.MethodsTo address these issues, this study introduces M3B-Net, a novel multi-modal, multi-branch framework that leverages fundus fluorescence angiography (FFA) images to improve retinal vessel segmentation in UWF fundus images. Specifically, M3B-Net tackles the low segmentation accuracy caused by the inherently low contrast of UWF fundus images. Additionally, we propose an enhanced UWF-based segmentation network in M3B-Net, specifically designed to improve the segmentation of fine retinal vessels. The segmentation network includes the Selective Fusion Module (SFM), which enhances feature extraction within the segmentation network by integrating features generated during the FFA imaging process. To further address the challenges of high-resolution UWF fundus images, we introduce a Local Perception Fusion Module (LPFM) to mitigate context loss during the segmentation cut-patch process. Complementing this, the Attention-Guided Upsampling Module (AUM) enhances segmentation performance through convolution operations guided by attention mechanisms.ResultsExtensive experimental evaluations demonstrate that our approach significantly outperforms existing state-of-the-art methods for UWF fundus image segmentation.https://www.frontiersin.org/articles/10.3389/fcell.2024.1532228/fullultra-widefieldfundus fluorescence angiographyretinal vessel segmentationmultimodal frameworkselective fusion |
| spellingShingle | Qihang Xie Qihang Xie Xuefei Li Yuanyuan Li Yuanyuan Li Jiayi Lu Jiayi Lu Shaodong Ma Yitian Zhao Yitian Zhao Jiong Zhang Jiong Zhang A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs Frontiers in Cell and Developmental Biology ultra-widefield fundus fluorescence angiography retinal vessel segmentation multimodal framework selective fusion |
| title | A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs |
| title_full | A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs |
| title_fullStr | A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs |
| title_full_unstemmed | A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs |
| title_short | A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs |
| title_sort | multi modal multi branch framework for retinal vessel segmentation using ultra widefield fundus photographs |
| topic | ultra-widefield fundus fluorescence angiography retinal vessel segmentation multimodal framework selective fusion |
| url | https://www.frontiersin.org/articles/10.3389/fcell.2024.1532228/full |
| work_keys_str_mv | AT qihangxie amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT qihangxie amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT xuefeili amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yuanyuanli amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yuanyuanli amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiayilu amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiayilu amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT shaodongma amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yitianzhao amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yitianzhao amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiongzhang amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiongzhang amultimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT qihangxie multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT qihangxie multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT xuefeili multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yuanyuanli multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yuanyuanli multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiayilu multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiayilu multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT shaodongma multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yitianzhao multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT yitianzhao multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiongzhang multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs AT jiongzhang multimodalmultibranchframeworkforretinalvesselsegmentationusingultrawidefieldfundusphotographs |