Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network
In recent years, weakly supervised semantic segmentation (WSSS) has garnered significant attention in remote sensing image analysis due to its low annotation cost. To address the issues of inaccurate and incomplete seed areas and unreliable pseudo masks in WSSS, we propose a novel WSSS method for re...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/17/5/808 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850053223594852352 |
|---|---|
| author | Zheng Chen Yuheng Lian Jing Bai Jingsen Zhang Zhu Xiao Biao Hou |
| author_facet | Zheng Chen Yuheng Lian Jing Bai Jingsen Zhang Zhu Xiao Biao Hou |
| author_sort | Zheng Chen |
| collection | DOAJ |
| description | In recent years, weakly supervised semantic segmentation (WSSS) has garnered significant attention in remote sensing image analysis due to its low annotation cost. To address the issues of inaccurate and incomplete seed areas and unreliable pseudo masks in WSSS, we propose a novel WSSS method for remote sensing images based on the Siamese Affinity Network (SAN) and the Segment Anything Model (SAM). First, we design a seed enhancement module for semantic affinity, which strengthens contextual relevance in the feature map by enforcing a unified constraint principle of cross-pixel similarity, thereby capturing semantically similar regions within the image. Second, leveraging the prior notion of cross-view consistency, we employ a Siamese network to regularize the consistency of CAMs from different affine-transformed images, providing additional supervision for weakly supervised learning. Finally, we utilize the SAM segmentation model to generate semantic superpixels, expanding the original CAM seeds to more completely and accurately extract target edges, thereby improving the quality of segmentation pseudo masks. Experimental results on the large-scale remote sensing datasets DRLSD and ISPRS Vaihingen demonstrate that our method achieves segmentation performance close to that of fully supervised semantic segmentation (FSSS) methods on both datasets. Ablation studies further verify the positive optimization effect of each module on segmentation pseudo labels. Our approach exhibits superior localization accuracy and precise visualization effects across different backbone networks, achieving state-of-the-art localization performance. |
| format | Article |
| id | doaj-art-a8f0444b614f4bdc97e105dfe32a6abb |
| institution | DOAJ |
| issn | 2072-4292 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-a8f0444b614f4bdc97e105dfe32a6abb2025-08-20T02:52:35ZengMDPI AGRemote Sensing2072-42922025-02-0117580810.3390/rs17050808Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity NetworkZheng Chen0Yuheng Lian1Jing Bai2Jingsen Zhang3Zhu Xiao4Biao Hou5The Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, School of Artificial Intelligence, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, School of Artificial Intelligence, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, School of Artificial Intelligence, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, School of Artificial Intelligence, Xidian University, Xi’an 710071, ChinaThe College of Computer Science and Electronic Engineering, Hunan University, Changsha 410082, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, School of Artificial Intelligence, Xidian University, Xi’an 710071, ChinaIn recent years, weakly supervised semantic segmentation (WSSS) has garnered significant attention in remote sensing image analysis due to its low annotation cost. To address the issues of inaccurate and incomplete seed areas and unreliable pseudo masks in WSSS, we propose a novel WSSS method for remote sensing images based on the Siamese Affinity Network (SAN) and the Segment Anything Model (SAM). First, we design a seed enhancement module for semantic affinity, which strengthens contextual relevance in the feature map by enforcing a unified constraint principle of cross-pixel similarity, thereby capturing semantically similar regions within the image. Second, leveraging the prior notion of cross-view consistency, we employ a Siamese network to regularize the consistency of CAMs from different affine-transformed images, providing additional supervision for weakly supervised learning. Finally, we utilize the SAM segmentation model to generate semantic superpixels, expanding the original CAM seeds to more completely and accurately extract target edges, thereby improving the quality of segmentation pseudo masks. Experimental results on the large-scale remote sensing datasets DRLSD and ISPRS Vaihingen demonstrate that our method achieves segmentation performance close to that of fully supervised semantic segmentation (FSSS) methods on both datasets. Ablation studies further verify the positive optimization effect of each module on segmentation pseudo labels. Our approach exhibits superior localization accuracy and precise visualization effects across different backbone networks, achieving state-of-the-art localization performance.https://www.mdpi.com/2072-4292/17/5/808Siamese networkSAMremote sensing imagesweakly supervised semantic segmentation (WSSS) |
| spellingShingle | Zheng Chen Yuheng Lian Jing Bai Jingsen Zhang Zhu Xiao Biao Hou Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network Remote Sensing Siamese network SAM remote sensing images weakly supervised semantic segmentation (WSSS) |
| title | Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network |
| title_full | Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network |
| title_fullStr | Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network |
| title_full_unstemmed | Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network |
| title_short | Weakly Supervised Semantic Segmentation of Remote Sensing Images Using Siamese Affinity Network |
| title_sort | weakly supervised semantic segmentation of remote sensing images using siamese affinity network |
| topic | Siamese network SAM remote sensing images weakly supervised semantic segmentation (WSSS) |
| url | https://www.mdpi.com/2072-4292/17/5/808 |
| work_keys_str_mv | AT zhengchen weaklysupervisedsemanticsegmentationofremotesensingimagesusingsiameseaffinitynetwork AT yuhenglian weaklysupervisedsemanticsegmentationofremotesensingimagesusingsiameseaffinitynetwork AT jingbai weaklysupervisedsemanticsegmentationofremotesensingimagesusingsiameseaffinitynetwork AT jingsenzhang weaklysupervisedsemanticsegmentationofremotesensingimagesusingsiameseaffinitynetwork AT zhuxiao weaklysupervisedsemanticsegmentationofremotesensingimagesusingsiameseaffinitynetwork AT biaohou weaklysupervisedsemanticsegmentationofremotesensingimagesusingsiameseaffinitynetwork |