Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images
Abstract Segmentation in computed tomography (CT) provides detailed anatomical information, while positron emission tomography (PET) provide the metabolic activity of cancer. Existing segmentation models in CT and PET either rely on early fusion, which struggles to effectively capture independent fe...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-04-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-95757-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849737312023347200 |
|---|---|
| author | Ibtihaj Ahmad Sadia Jabbar Anwar Bagh Hussain Atiq ur Rehman Amine Bermak |
| author_facet | Ibtihaj Ahmad Sadia Jabbar Anwar Bagh Hussain Atiq ur Rehman Amine Bermak |
| author_sort | Ibtihaj Ahmad |
| collection | DOAJ |
| description | Abstract Segmentation in computed tomography (CT) provides detailed anatomical information, while positron emission tomography (PET) provide the metabolic activity of cancer. Existing segmentation models in CT and PET either rely on early fusion, which struggles to effectively capture independent features from each modality, or late fusion, which is computationally expensive and fails to leverage the complementary nature of the two modalities. This research addresses the gap by proposing an intermediate fusion approach that optimally balances the strengths of both modalities. Our method leverages anatomical features to guide the fusion process while preserving spatial representation quality. We achieve this through the separate encoding of anatomical and metabolic features followed by an attentive fusion decoder. Unlike traditional fixed normalization techniques, we introduce novel “zero layers” with learnable normalization. The proposed intermediate fusion reduces the number of filters, resulting in a lightweight model. Our approach demonstrates superior performance, achieving a dice score of 0.8184 and an $$\hbox {HD}^{95}$$ score of 2.31. The implications of this study include more precise tumor delineation, leading to enhanced cancer diagnosis and more effective treatment planning. |
| format | Article |
| id | doaj-art-88c6a12e7e0f407aa465aebc92b72e87 |
| institution | DOAJ |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-88c6a12e7e0f407aa465aebc92b72e872025-08-20T03:06:57ZengNature PortfolioScientific Reports2045-23222025-04-0115111310.1038/s41598-025-95757-6Anatomy guided modality fusion for cancer segmentation in PET CT volumes and imagesIbtihaj Ahmad0Sadia Jabbar Anwar1Bagh Hussain2Atiq ur Rehman3Amine Bermak4Northwestern Polytechnical UniversityNorthwestern Polytechnical UniversityNorthwestern Polytechnical UniversityDivision of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa UniversityDivision of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa UniversityAbstract Segmentation in computed tomography (CT) provides detailed anatomical information, while positron emission tomography (PET) provide the metabolic activity of cancer. Existing segmentation models in CT and PET either rely on early fusion, which struggles to effectively capture independent features from each modality, or late fusion, which is computationally expensive and fails to leverage the complementary nature of the two modalities. This research addresses the gap by proposing an intermediate fusion approach that optimally balances the strengths of both modalities. Our method leverages anatomical features to guide the fusion process while preserving spatial representation quality. We achieve this through the separate encoding of anatomical and metabolic features followed by an attentive fusion decoder. Unlike traditional fixed normalization techniques, we introduce novel “zero layers” with learnable normalization. The proposed intermediate fusion reduces the number of filters, resulting in a lightweight model. Our approach demonstrates superior performance, achieving a dice score of 0.8184 and an $$\hbox {HD}^{95}$$ score of 2.31. The implications of this study include more precise tumor delineation, leading to enhanced cancer diagnosis and more effective treatment planning.https://doi.org/10.1038/s41598-025-95757-6 |
| spellingShingle | Ibtihaj Ahmad Sadia Jabbar Anwar Bagh Hussain Atiq ur Rehman Amine Bermak Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images Scientific Reports |
| title | Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images |
| title_full | Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images |
| title_fullStr | Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images |
| title_full_unstemmed | Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images |
| title_short | Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images |
| title_sort | anatomy guided modality fusion for cancer segmentation in pet ct volumes and images |
| url | https://doi.org/10.1038/s41598-025-95757-6 |
| work_keys_str_mv | AT ibtihajahmad anatomyguidedmodalityfusionforcancersegmentationinpetctvolumesandimages AT sadiajabbaranwar anatomyguidedmodalityfusionforcancersegmentationinpetctvolumesandimages AT baghhussain anatomyguidedmodalityfusionforcancersegmentationinpetctvolumesandimages AT atiqurrehman anatomyguidedmodalityfusionforcancersegmentationinpetctvolumesandimages AT aminebermak anatomyguidedmodalityfusionforcancersegmentationinpetctvolumesandimages |