ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation

In the realm of artificial intelligence, the emergence of foundation models, backed by high computing capabilities and extensive data, has been revolutionary. A segment anything model (SAM), built on the vision transformer (ViT) model with millions of parameters and trained on its corresponding larg...

Full description

Saved in:
Bibliographic Details
Main Authors: Xinyang Pu, Hecheng Jia, Linghao Zheng, Feng Wang, Feng Xu
Format: Article
Language:English
Published: IEEE 2025-01-01
Series:IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Subjects:
Online Access:https://ieeexplore.ieee.org/document/10849617/
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1823859652967792640
author Xinyang Pu
Hecheng Jia
Linghao Zheng
Feng Wang
Feng Xu
author_facet Xinyang Pu
Hecheng Jia
Linghao Zheng
Feng Wang
Feng Xu
author_sort Xinyang Pu
collection DOAJ
description In the realm of artificial intelligence, the emergence of foundation models, backed by high computing capabilities and extensive data, has been revolutionary. A segment anything model (SAM), built on the vision transformer (ViT) model with millions of parameters and trained on its corresponding large-scale dataset SA-1B, excels in various segmentation scenarios relying on its significance of semantic information and generalization ability. Such achievement of visual foundation model stimulates continuous researches on specific downstream tasks in computer vision. The classwise-SAM-adapter (CWSAM) is designed to adapt the high-performing SAM for landcover classification on space-borne synthetic aperture radar (SAR) images. The proposed CWSAM freezes most of SAM's parameters and incorporates lightweight adapters for parameter-efficient fine-tuning, and a classwise mask decoder is designed to achieve semantic segmentation task. This adapt-tuning method allows for efficient landcover classification of SAR images, balancing the accuracy with computational demand. In addition, the task-specific input module injects low-frequency information of SAR images by MLP-based layers to improve the model performance. Compared to conventional state-of-the-art semantic segmentation algorithms by extensive experiments, CWSAM showcases enhanced performance with fewer computing resources, highlighting the potential of leveraging foundational models such as SAM for specific downstream tasks in the SAR domain.
format Article
id doaj-art-eee3614af7ad45229ee58ec3621ceb51
institution Kabale University
issn 1939-1404
2151-1535
language English
publishDate 2025-01-01
publisher IEEE
record_format Article
series IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
spelling doaj-art-eee3614af7ad45229ee58ec3621ceb512025-02-11T00:00:27ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01184791480410.1109/JSTARS.2025.353269010849617ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic SegmentationXinyang Pu0https://orcid.org/0009-0002-0627-4603Hecheng Jia1https://orcid.org/0000-0001-7538-4094Linghao Zheng2Feng Wang3https://orcid.org/0000-0002-2378-9126Feng Xu4https://orcid.org/0000-0002-7015-1467Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, ChinaKey Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, ChinaKey Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, ChinaKey Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, ChinaKey Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, ChinaIn the realm of artificial intelligence, the emergence of foundation models, backed by high computing capabilities and extensive data, has been revolutionary. A segment anything model (SAM), built on the vision transformer (ViT) model with millions of parameters and trained on its corresponding large-scale dataset SA-1B, excels in various segmentation scenarios relying on its significance of semantic information and generalization ability. Such achievement of visual foundation model stimulates continuous researches on specific downstream tasks in computer vision. The classwise-SAM-adapter (CWSAM) is designed to adapt the high-performing SAM for landcover classification on space-borne synthetic aperture radar (SAR) images. The proposed CWSAM freezes most of SAM's parameters and incorporates lightweight adapters for parameter-efficient fine-tuning, and a classwise mask decoder is designed to achieve semantic segmentation task. This adapt-tuning method allows for efficient landcover classification of SAR images, balancing the accuracy with computational demand. In addition, the task-specific input module injects low-frequency information of SAR images by MLP-based layers to improve the model performance. Compared to conventional state-of-the-art semantic segmentation algorithms by extensive experiments, CWSAM showcases enhanced performance with fewer computing resources, highlighting the potential of leveraging foundational models such as SAM for specific downstream tasks in the SAR domain.https://ieeexplore.ieee.org/document/10849617/Adapter tuninglandcover classificationparameter-efficient fine-tuningsegment anything (SA)synthetic aperture radar (SAR)visual foundation model
spellingShingle Xinyang Pu
Hecheng Jia
Linghao Zheng
Feng Wang
Feng Xu
ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Adapter tuning
landcover classification
parameter-efficient fine-tuning
segment anything (SA)
synthetic aperture radar (SAR)
visual foundation model
title ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation
title_full ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation
title_fullStr ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation
title_full_unstemmed ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation
title_short ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation
title_sort classwise sam adapter parameter efficient fine tuning adapts segment anything to sar domain for semantic segmentation
topic Adapter tuning
landcover classification
parameter-efficient fine-tuning
segment anything (SA)
synthetic aperture radar (SAR)
visual foundation model
url https://ieeexplore.ieee.org/document/10849617/
work_keys_str_mv AT xinyangpu classwisesamadapterparameterefficientfinetuningadaptssegmentanythingtosardomainforsemanticsegmentation
AT hechengjia classwisesamadapterparameterefficientfinetuningadaptssegmentanythingtosardomainforsemanticsegmentation
AT linghaozheng classwisesamadapterparameterefficientfinetuningadaptssegmentanythingtosardomainforsemanticsegmentation
AT fengwang classwisesamadapterparameterefficientfinetuningadaptssegmentanythingtosardomainforsemanticsegmentation
AT fengxu classwisesamadapterparameterefficientfinetuningadaptssegmentanythingtosardomainforsemanticsegmentation