Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation
Abstract BackgroundMedical image segmentation is crucial for diagnosis and treatment planning in radiology, but it traditionally requires extensive manual effort and specialized training data. With its novel video tracking capabilities, the Segment Anything Model 2 (SAM 2) pre...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-04-01
|
| Series: | JMIR AI |
| Online Access: | https://ai.jmir.org/2025/1/e72109 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850086448389160960 |
|---|---|
| author | Yosuke Yamagishi Shouhei Hanaoka Tomohiro Kikuchi Takahiro Nakao Yuta Nakamura Yukihiro Nomura Soichiro Miki Takeharu Yoshikawa Osamu Abe |
| author_facet | Yosuke Yamagishi Shouhei Hanaoka Tomohiro Kikuchi Takahiro Nakao Yuta Nakamura Yukihiro Nomura Soichiro Miki Takeharu Yoshikawa Osamu Abe |
| author_sort | Yosuke Yamagishi |
| collection | DOAJ |
| description |
Abstract
BackgroundMedical image segmentation is crucial for diagnosis and treatment planning in radiology, but it traditionally requires extensive manual effort and specialized training data. With its novel video tracking capabilities, the Segment Anything Model 2 (SAM 2) presents a potential solution for automated 3D medical image segmentation without the need for domain-specific training. However, its effectiveness in medical applications, particularly in abdominal computed tomography (CT) imaging remains unexplored.
ObjectiveThe aim of this study was to evaluate the zero-shot performance of SAM 2 in 3D segmentation of abdominal organs in CT scans and to investigate the effects of prompt settings on segmentation results.
MethodsIn this retrospective study, we used a subset of the TotalSegmentator CT dataset from eight institutions to assess SAM 2’s ability to segment eight abdominal organs. Segmentation was initiated from three different z-coordinate levels (caudal, mid, and cranial levels) of each organ. Performance was measured using the dice similarity coefficient (DSC). We also analyzed the impact of “negative prompts,” which explicitly exclude certain regions from the segmentation process, on accuracy.
ResultsA total of 123 patients (mean age 60.7, SD 15.5 years; 63 men, 60 women) were evaluated. As a zero-shot approach, larger organs with clear boundaries demonstrated high segmentation performance, with mean DSCs as follows: liver, 0.821 (SD 0.192); right kidney, 0.862 (SD 0.212); left kidney, 0.870 (SD 0.154); and spleen, 0.891 (SD 0.131). Smaller organs showed lower performance: gallbladder, 0.531 (SD 0.291); pancreas, 0.361 (SD 0.197); and adrenal glands—right, 0.203 (SD 0.222) and left, 0.308 (SD 0.234). The initial slice for segmentation and the use of negative prompts significantly influenced the results. By removing negative prompts from the input, the DSCs significantly decreased for six organs.
ConclusionsSAM 2 demonstrated promising zero-shot performance in segmenting certain abdominal organs in CT scans, particularly larger organs. Performance was significantly influenced by input negative prompts and initial slice selection, highlighting the importance of optimizing these factors. |
| format | Article |
| id | doaj-art-e3a56e849cd54180850f10a0d9de0466 |
| institution | DOAJ |
| issn | 2817-1705 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR AI |
| spelling | doaj-art-e3a56e849cd54180850f10a0d9de04662025-08-20T02:43:28ZengJMIR PublicationsJMIR AI2817-17052025-04-014e72109e7210910.2196/72109Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and ValidationYosuke Yamagishihttp://orcid.org/0009-0006-7688-3075Shouhei Hanaokahttp://orcid.org/0000-0002-7496-1651Tomohiro Kikuchihttp://orcid.org/0000-0002-4222-4569Takahiro Nakaohttp://orcid.org/0000-0001-9498-7501Yuta Nakamurahttp://orcid.org/0000-0001-6962-6704Yukihiro Nomurahttp://orcid.org/0000-0001-6471-9936Soichiro Mikihttp://orcid.org/0000-0001-6365-6531Takeharu Yoshikawahttp://orcid.org/0000-0002-1924-5468Osamu Abehttp://orcid.org/0000-0002-1180-2629 Abstract BackgroundMedical image segmentation is crucial for diagnosis and treatment planning in radiology, but it traditionally requires extensive manual effort and specialized training data. With its novel video tracking capabilities, the Segment Anything Model 2 (SAM 2) presents a potential solution for automated 3D medical image segmentation without the need for domain-specific training. However, its effectiveness in medical applications, particularly in abdominal computed tomography (CT) imaging remains unexplored. ObjectiveThe aim of this study was to evaluate the zero-shot performance of SAM 2 in 3D segmentation of abdominal organs in CT scans and to investigate the effects of prompt settings on segmentation results. MethodsIn this retrospective study, we used a subset of the TotalSegmentator CT dataset from eight institutions to assess SAM 2’s ability to segment eight abdominal organs. Segmentation was initiated from three different z-coordinate levels (caudal, mid, and cranial levels) of each organ. Performance was measured using the dice similarity coefficient (DSC). We also analyzed the impact of “negative prompts,” which explicitly exclude certain regions from the segmentation process, on accuracy. ResultsA total of 123 patients (mean age 60.7, SD 15.5 years; 63 men, 60 women) were evaluated. As a zero-shot approach, larger organs with clear boundaries demonstrated high segmentation performance, with mean DSCs as follows: liver, 0.821 (SD 0.192); right kidney, 0.862 (SD 0.212); left kidney, 0.870 (SD 0.154); and spleen, 0.891 (SD 0.131). Smaller organs showed lower performance: gallbladder, 0.531 (SD 0.291); pancreas, 0.361 (SD 0.197); and adrenal glands—right, 0.203 (SD 0.222) and left, 0.308 (SD 0.234). The initial slice for segmentation and the use of negative prompts significantly influenced the results. By removing negative prompts from the input, the DSCs significantly decreased for six organs. ConclusionsSAM 2 demonstrated promising zero-shot performance in segmenting certain abdominal organs in CT scans, particularly larger organs. Performance was significantly influenced by input negative prompts and initial slice selection, highlighting the importance of optimizing these factors.https://ai.jmir.org/2025/1/e72109 |
| spellingShingle | Yosuke Yamagishi Shouhei Hanaoka Tomohiro Kikuchi Takahiro Nakao Yuta Nakamura Yukihiro Nomura Soichiro Miki Takeharu Yoshikawa Osamu Abe Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation JMIR AI |
| title | Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation |
| title_full | Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation |
| title_fullStr | Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation |
| title_full_unstemmed | Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation |
| title_short | Using Segment Anything Model 2 for Zero-Shot 3D Segmentation of Abdominal Organs in Computed Tomography Scans to Adapt Video Tracking Capabilities for 3D Medical Imaging: Algorithm Development and Validation |
| title_sort | using segment anything model 2 for zero shot 3d segmentation of abdominal organs in computed tomography scans to adapt video tracking capabilities for 3d medical imaging algorithm development and validation |
| url | https://ai.jmir.org/2025/1/e72109 |
| work_keys_str_mv | AT yosukeyamagishi usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT shouheihanaoka usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT tomohirokikuchi usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT takahironakao usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT yutanakamura usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT yukihironomura usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT soichiromiki usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT takeharuyoshikawa usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation AT osamuabe usingsegmentanythingmodel2forzeroshot3dsegmentationofabdominalorgansincomputedtomographyscanstoadaptvideotrackingcapabilitiesfor3dmedicalimagingalgorithmdevelopmentandvalidation |