Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising
Abstract Cell instances segmentation in cervical cytology images have crucial importance for cancer screening and automatic biomedical image analysis. However, that the highly overlapping cells cause ambiguous boundaries makes accurate cell segmentation still a challenging task. This paper proposes...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-05-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-02379-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849325818448183296 |
|---|---|
| author | Baocan Zhang Wei Zhao Chenxi Huang Quan Lan Weihong Lu |
| author_facet | Baocan Zhang Wei Zhao Chenxi Huang Quan Lan Weihong Lu |
| author_sort | Baocan Zhang |
| collection | DOAJ |
| description | Abstract Cell instances segmentation in cervical cytology images have crucial importance for cancer screening and automatic biomedical image analysis. However, that the highly overlapping cells cause ambiguous boundaries makes accurate cell segmentation still a challenging task. This paper proposes a novel transformer-based neural network to segment overlapping cervical cells, by using the technique of denoising with Mask2Former. The model optimization includes two aspects. One is that class embeddings of ground truth categories are used as extra content queries to transformer decoder. The other is that noised ground truth masks are fed into each layer of transformer decoder and then the model is trained to reconstruct the original masks. Compared with the best result on ISBI2014 dataset, the proposed model obtains performance improvement by $$3.4\%$$ on DSC, $$2.3\%$$ on TPRp and $$2\%$$ on FNRo, respectively. Especially, the masks predicted by our model are more precise than other methods and our previous models. Those results show that the proposed method could be effective in segmenting overlapping cells and help pathologists detect cell lesions. |
| format | Article |
| id | doaj-art-ab9f01b34cdb441aa8744a9f923d8255 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-ab9f01b34cdb441aa8744a9f923d82552025-08-20T03:48:18ZengNature PortfolioScientific Reports2045-23222025-05-0115111310.1038/s41598-025-02379-zOptimized segmentation of overlapping cervical cells based on Mask2Former and denoisingBaocan Zhang0Wei Zhao1Chenxi Huang2Quan Lan3Weihong Lu4Chengyi College, Jimei UniversityChengyi College, Jimei UniversitySchool of Informatics, Xiamen UniversityDepartment of Neurology, First Affiliated Hospital of Xiamen UniversityDepartment of Gynecology, Zhongshan Hospital (Xiamen), Fudan UniversityAbstract Cell instances segmentation in cervical cytology images have crucial importance for cancer screening and automatic biomedical image analysis. However, that the highly overlapping cells cause ambiguous boundaries makes accurate cell segmentation still a challenging task. This paper proposes a novel transformer-based neural network to segment overlapping cervical cells, by using the technique of denoising with Mask2Former. The model optimization includes two aspects. One is that class embeddings of ground truth categories are used as extra content queries to transformer decoder. The other is that noised ground truth masks are fed into each layer of transformer decoder and then the model is trained to reconstruct the original masks. Compared with the best result on ISBI2014 dataset, the proposed model obtains performance improvement by $$3.4\%$$ on DSC, $$2.3\%$$ on TPRp and $$2\%$$ on FNRo, respectively. Especially, the masks predicted by our model are more precise than other methods and our previous models. Those results show that the proposed method could be effective in segmenting overlapping cells and help pathologists detect cell lesions.https://doi.org/10.1038/s41598-025-02379-zMask2FormerDenoisingOverlapping cervical cellsNoised masksISBI |
| spellingShingle | Baocan Zhang Wei Zhao Chenxi Huang Quan Lan Weihong Lu Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising Scientific Reports Mask2Former Denoising Overlapping cervical cells Noised masks ISBI |
| title | Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising |
| title_full | Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising |
| title_fullStr | Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising |
| title_full_unstemmed | Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising |
| title_short | Optimized segmentation of overlapping cervical cells based on Mask2Former and denoising |
| title_sort | optimized segmentation of overlapping cervical cells based on mask2former and denoising |
| topic | Mask2Former Denoising Overlapping cervical cells Noised masks ISBI |
| url | https://doi.org/10.1038/s41598-025-02379-z |
| work_keys_str_mv | AT baocanzhang optimizedsegmentationofoverlappingcervicalcellsbasedonmask2formeranddenoising AT weizhao optimizedsegmentationofoverlappingcervicalcellsbasedonmask2formeranddenoising AT chenxihuang optimizedsegmentationofoverlappingcervicalcellsbasedonmask2formeranddenoising AT quanlan optimizedsegmentationofoverlappingcervicalcellsbasedonmask2formeranddenoising AT weihonglu optimizedsegmentationofoverlappingcervicalcellsbasedonmask2formeranddenoising |