Instance-level semantic segmentation of nuclei based on multimodal structure encoding
Abstract Background Accurate segmentation and classification of cell nuclei are crucial for histopathological image analysis. However, existing deep neural network-based methods often struggle to capture complex morphological features and global spatial distributions of cell nuclei due to their reli...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-02-01
|
| Series: | BMC Bioinformatics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12859-025-06066-8 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823861550459387904 |
|---|---|
| author | Bo Guan Guangdi Chu Ziying Wang Jianmin Li Bo Yi |
| author_facet | Bo Guan Guangdi Chu Ziying Wang Jianmin Li Bo Yi |
| author_sort | Bo Guan |
| collection | DOAJ |
| description | Abstract Background Accurate segmentation and classification of cell nuclei are crucial for histopathological image analysis. However, existing deep neural network-based methods often struggle to capture complex morphological features and global spatial distributions of cell nuclei due to their reliance on local receptive fields. Methods This study proposes a graph neural structure encoding framework based on a vision-language model. The framework incorporates: (1) A multi-scale feature fusion and knowledge distillation module utilizing the Contrastive Language-Image Pre-training (CLIP) model’s image encoder; (2) A method to transform morphological features of cells into textual descriptions for semantic representation; and (3) A graph neural network approach to learn spatial relationships and contextual information between cell nuclei. Results Experimental results demonstrate that the proposed method significantly improves the accuracy of cell nucleus segmentation and classification compared to existing approaches. The framework effectively captures complex nuclear structures and global distribution features, leading to enhanced performance in histopathological image analysis. Conclusions By deeply mining the morphological features of cell nuclei and their spatial topological relationships, our graph neural structure encoding framework achieves high-precision nuclear segmentation and classification. This approach shows significant potential for enhancing histopathological image analysis, potentially leading to more accurate diagnoses and improved understanding of cellular structures in pathological tissues. |
| format | Article |
| id | doaj-art-4c46d88a7eaf448b99a348f49ec2b09c |
| institution | Kabale University |
| issn | 1471-2105 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Bioinformatics |
| spelling | doaj-art-4c46d88a7eaf448b99a348f49ec2b09c2025-02-09T12:56:57ZengBMCBMC Bioinformatics1471-21052025-02-0126112510.1186/s12859-025-06066-8Instance-level semantic segmentation of nuclei based on multimodal structure encodingBo Guan0Guangdi Chu1Ziying Wang2Jianmin Li3Bo Yi4Key Lab for Mechanism Theory and Equipment Design of Ministry of Education, Tianjin UniversityDepartment of Urology, The Affiliated Hospital of Qingdao UniversityDepartment of Medicine, Qingdao UniversityKey Lab for Mechanism Theory and Equipment Design of Ministry of Education, Tianjin UniversityDepartment of General Surgery, Third Xiangya Hospital, Central South UniversityAbstract Background Accurate segmentation and classification of cell nuclei are crucial for histopathological image analysis. However, existing deep neural network-based methods often struggle to capture complex morphological features and global spatial distributions of cell nuclei due to their reliance on local receptive fields. Methods This study proposes a graph neural structure encoding framework based on a vision-language model. The framework incorporates: (1) A multi-scale feature fusion and knowledge distillation module utilizing the Contrastive Language-Image Pre-training (CLIP) model’s image encoder; (2) A method to transform morphological features of cells into textual descriptions for semantic representation; and (3) A graph neural network approach to learn spatial relationships and contextual information between cell nuclei. Results Experimental results demonstrate that the proposed method significantly improves the accuracy of cell nucleus segmentation and classification compared to existing approaches. The framework effectively captures complex nuclear structures and global distribution features, leading to enhanced performance in histopathological image analysis. Conclusions By deeply mining the morphological features of cell nuclei and their spatial topological relationships, our graph neural structure encoding framework achieves high-precision nuclear segmentation and classification. This approach shows significant potential for enhancing histopathological image analysis, potentially leading to more accurate diagnoses and improved understanding of cellular structures in pathological tissues.https://doi.org/10.1186/s12859-025-06066-8Cell nucleus segmentationHistopathological imageGraph neural networksMultimodal fusion |
| spellingShingle | Bo Guan Guangdi Chu Ziying Wang Jianmin Li Bo Yi Instance-level semantic segmentation of nuclei based on multimodal structure encoding BMC Bioinformatics Cell nucleus segmentation Histopathological image Graph neural networks Multimodal fusion |
| title | Instance-level semantic segmentation of nuclei based on multimodal structure encoding |
| title_full | Instance-level semantic segmentation of nuclei based on multimodal structure encoding |
| title_fullStr | Instance-level semantic segmentation of nuclei based on multimodal structure encoding |
| title_full_unstemmed | Instance-level semantic segmentation of nuclei based on multimodal structure encoding |
| title_short | Instance-level semantic segmentation of nuclei based on multimodal structure encoding |
| title_sort | instance level semantic segmentation of nuclei based on multimodal structure encoding |
| topic | Cell nucleus segmentation Histopathological image Graph neural networks Multimodal fusion |
| url | https://doi.org/10.1186/s12859-025-06066-8 |
| work_keys_str_mv | AT boguan instancelevelsemanticsegmentationofnucleibasedonmultimodalstructureencoding AT guangdichu instancelevelsemanticsegmentationofnucleibasedonmultimodalstructureencoding AT ziyingwang instancelevelsemanticsegmentationofnucleibasedonmultimodalstructureencoding AT jianminli instancelevelsemanticsegmentationofnucleibasedonmultimodalstructureencoding AT boyi instancelevelsemanticsegmentationofnucleibasedonmultimodalstructureencoding |