Tuning vision foundation models for rectal cancer segmentation from CT scans
Abstract Background Rectal cancer segmentation in CT is crucial for timely diagnosis. Despite promising methods, challenges remain due to the rectum’s complex anatomy and the lack of a comprehensive annotated dataset. Methods A total of 33,024 slice pairs from 398 rectal cancer patients in a new sou...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Communications Medicine |
| Online Access: | https://doi.org/10.1038/s43856-025-00953-0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849761641806168064 |
|---|---|
| author | Hantao Zhang Weidong Guo Shouhong Wan Bingbing Zou Wanqin Wang Chenyang Qiu Kaige Liu Peiquan Jin Jiancheng Yang |
| author_facet | Hantao Zhang Weidong Guo Shouhong Wan Bingbing Zou Wanqin Wang Chenyang Qiu Kaige Liu Peiquan Jin Jiancheng Yang |
| author_sort | Hantao Zhang |
| collection | DOAJ |
| description | Abstract Background Rectal cancer segmentation in CT is crucial for timely diagnosis. Despite promising methods, challenges remain due to the rectum’s complex anatomy and the lack of a comprehensive annotated dataset. Methods A total of 33,024 slice pairs from 398 rectal cancer patients in a new source center are enrolled into our dataset, named CARE Dataset, with pixel-level annotations for both normal and cancerous rectum tissue. We split it into 317 cases for training and 81 for testing. Additionally, we introduce a segmentation model, U-SAM, which, to the best of our knowledge, is a novel approach designed to handle the complex anatomy of the rectum by incorporating prompt information. Segmentation performance for both normal and cancerous rectum was evaluated using Intersection-over-Union (IoU), Dice Coefficient (Dice), and Normalized Surface Distance (NSD). With the assistance of 46 clinical practitioners, an observer study is conducted to benchmark the U-SAM with human performance and evaluate its clinical applicability. The original new source 398 CT scans and our code are openly available for research. Results Our method achieves Dice of 71.23% for normal rectum and 76.38% for rectal tumor, with IoU of 55.32% and 61.78%, and NSD values of 83.63% and 58.59%, respectively, surpassing state-of-the-art methods. The observer study validates that U-SAM can produce diagnostic results comparable to those of highly experienced doctors in just 3 seconds of inference time (with about 5 minutes for prompt acquisition) in clinical settings. Conclusions The proposed U-SAM offers an efficient and reliable method for segmenting rectal cancer and normal tissue, significantly reducing time in clinical settings and effectively assisting radiologists. We believe this initial exploration in CT-based rectal cancer segmentation will be instrumental for future diagnosis. |
| format | Article |
| id | doaj-art-e3f7aee5407c4a96a43c8c6d602a7ae8 |
| institution | DOAJ |
| issn | 2730-664X |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Medicine |
| spelling | doaj-art-e3f7aee5407c4a96a43c8c6d602a7ae82025-08-20T03:05:57ZengNature PortfolioCommunications Medicine2730-664X2025-07-015111110.1038/s43856-025-00953-0Tuning vision foundation models for rectal cancer segmentation from CT scansHantao Zhang0Weidong Guo1Shouhong Wan2Bingbing Zou3Wanqin Wang4Chenyang Qiu5Kaige Liu6Peiquan Jin7Jiancheng Yang8School of Computer Science and Technology, University of Science and Technology of ChinaSchool of Computer Science and Technology, University of Science and Technology of ChinaSchool of Computer Science and Technology, University of Science and Technology of ChinaInstitute of Artificial Intelligence, Hefei Comprehensive National Science CenterInstitute of Artificial Intelligence, Hefei Comprehensive National Science CenterInstitute of Artificial Intelligence, Hefei Comprehensive National Science CenterInstitute of Artificial Intelligence, Hefei Comprehensive National Science CenterSchool of Computer Science and Technology, University of Science and Technology of ChinaComputer Vision Laboratory, Swiss Federal Institute of Technology Lausanne (EPFL)Abstract Background Rectal cancer segmentation in CT is crucial for timely diagnosis. Despite promising methods, challenges remain due to the rectum’s complex anatomy and the lack of a comprehensive annotated dataset. Methods A total of 33,024 slice pairs from 398 rectal cancer patients in a new source center are enrolled into our dataset, named CARE Dataset, with pixel-level annotations for both normal and cancerous rectum tissue. We split it into 317 cases for training and 81 for testing. Additionally, we introduce a segmentation model, U-SAM, which, to the best of our knowledge, is a novel approach designed to handle the complex anatomy of the rectum by incorporating prompt information. Segmentation performance for both normal and cancerous rectum was evaluated using Intersection-over-Union (IoU), Dice Coefficient (Dice), and Normalized Surface Distance (NSD). With the assistance of 46 clinical practitioners, an observer study is conducted to benchmark the U-SAM with human performance and evaluate its clinical applicability. The original new source 398 CT scans and our code are openly available for research. Results Our method achieves Dice of 71.23% for normal rectum and 76.38% for rectal tumor, with IoU of 55.32% and 61.78%, and NSD values of 83.63% and 58.59%, respectively, surpassing state-of-the-art methods. The observer study validates that U-SAM can produce diagnostic results comparable to those of highly experienced doctors in just 3 seconds of inference time (with about 5 minutes for prompt acquisition) in clinical settings. Conclusions The proposed U-SAM offers an efficient and reliable method for segmenting rectal cancer and normal tissue, significantly reducing time in clinical settings and effectively assisting radiologists. We believe this initial exploration in CT-based rectal cancer segmentation will be instrumental for future diagnosis.https://doi.org/10.1038/s43856-025-00953-0 |
| spellingShingle | Hantao Zhang Weidong Guo Shouhong Wan Bingbing Zou Wanqin Wang Chenyang Qiu Kaige Liu Peiquan Jin Jiancheng Yang Tuning vision foundation models for rectal cancer segmentation from CT scans Communications Medicine |
| title | Tuning vision foundation models for rectal cancer segmentation from CT scans |
| title_full | Tuning vision foundation models for rectal cancer segmentation from CT scans |
| title_fullStr | Tuning vision foundation models for rectal cancer segmentation from CT scans |
| title_full_unstemmed | Tuning vision foundation models for rectal cancer segmentation from CT scans |
| title_short | Tuning vision foundation models for rectal cancer segmentation from CT scans |
| title_sort | tuning vision foundation models for rectal cancer segmentation from ct scans |
| url | https://doi.org/10.1038/s43856-025-00953-0 |
| work_keys_str_mv | AT hantaozhang tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT weidongguo tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT shouhongwan tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT bingbingzou tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT wanqinwang tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT chenyangqiu tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT kaigeliu tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT peiquanjin tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans AT jianchengyang tuningvisionfoundationmodelsforrectalcancersegmentationfromctscans |