Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement
Remote sensing image segmentation is a fundamental task in Earth observation. Rapid development has been made in the past decade owing to the deep learning techniques. Most of the existing methods assume that the training and inference remote sensing images hold the identical and independent distrib...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10999045/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850231985499275264 |
|---|---|
| author | Jie Luo Tianwen Luo Maoyang Wang Linyi Li Wen Zhang Lingkui Meng |
| author_facet | Jie Luo Tianwen Luo Maoyang Wang Linyi Li Wen Zhang Lingkui Meng |
| author_sort | Jie Luo |
| collection | DOAJ |
| description | Remote sensing image segmentation is a fundamental task in Earth observation. Rapid development has been made in the past decade owing to the deep learning techniques. Most of the existing methods assume that the training and inference remote sensing images hold the identical and independent distribution. However, in practical cases, the pretrained segmentation model has to deal with remote sensing images that have not seen before. In this article, we advance this bottleneck by introducing the domain generalization (DG) paradigm to remote sensing image segmentation. It does not need to involve any images from the target domain during training, compared with the existing domain adaptation (DA) paradigm. This article focuses on the key challenges from the various instance size and dramatic image appearance from cross-domain remote sensing images, and proposes a novel multiscale instance disentanglement learning scheme. Technically, it consists of three key components, namely, multiscale instance encoding (MSIE), domain-invariant instance disentanglement (DII), and domain generalized semantic decoding (SD). The proposed MSIE module incorporates both depth convolution and multiscale representing, so as to learn robust semantic representation despite the cross-domain scale variation. The DII module, on the other hand, helps eliminate the impact of cross-domain styles. Finally, the domain generalized SD fuses the style-invariant representation and outputs the per-pixel semantic prediction. Extensive experiments on four cross-domain settings show its effectiveness over existing DA and DG methods from both remote sensing and computer vision community, yielding an mean intersection over union improvement of 3.3% and 6.0% on the Postdam and Vaihigen unseen target domain over the second best, respectively. |
| format | Article |
| id | doaj-art-55c96c2e6dfc43c4b2cb1120f55419c7 |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-55c96c2e6dfc43c4b2cb1120f55419c72025-08-20T02:03:19ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118135381355310.1109/JSTARS.2025.356885310999045Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance DisentanglementJie Luo0https://orcid.org/0009-0006-6847-6939Tianwen Luo1Maoyang Wang2Linyi Li3https://orcid.org/0000-0002-2185-8407Wen Zhang4https://orcid.org/0000-0001-5576-5973Lingkui Meng5https://orcid.org/0000-0002-4097-3080School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaGuiZhou Water and Power Survey-Design Institute Company Ltd., Guiyang, ChinaGuiZhou Water and Power Survey-Design Institute Company Ltd., Guiyang, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaRemote sensing image segmentation is a fundamental task in Earth observation. Rapid development has been made in the past decade owing to the deep learning techniques. Most of the existing methods assume that the training and inference remote sensing images hold the identical and independent distribution. However, in practical cases, the pretrained segmentation model has to deal with remote sensing images that have not seen before. In this article, we advance this bottleneck by introducing the domain generalization (DG) paradigm to remote sensing image segmentation. It does not need to involve any images from the target domain during training, compared with the existing domain adaptation (DA) paradigm. This article focuses on the key challenges from the various instance size and dramatic image appearance from cross-domain remote sensing images, and proposes a novel multiscale instance disentanglement learning scheme. Technically, it consists of three key components, namely, multiscale instance encoding (MSIE), domain-invariant instance disentanglement (DII), and domain generalized semantic decoding (SD). The proposed MSIE module incorporates both depth convolution and multiscale representing, so as to learn robust semantic representation despite the cross-domain scale variation. The DII module, on the other hand, helps eliminate the impact of cross-domain styles. Finally, the domain generalized SD fuses the style-invariant representation and outputs the per-pixel semantic prediction. Extensive experiments on four cross-domain settings show its effectiveness over existing DA and DG methods from both remote sensing and computer vision community, yielding an mean intersection over union improvement of 3.3% and 6.0% on the Postdam and Vaihigen unseen target domain over the second best, respectively.https://ieeexplore.ieee.org/document/10999045/Domain generalization (DG)instance disentanglementmultiscale representationremote sensing image segmentation |
| spellingShingle | Jie Luo Tianwen Luo Maoyang Wang Linyi Li Wen Zhang Lingkui Meng Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Domain generalization (DG) instance disentanglement multiscale representation remote sensing image segmentation |
| title | Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement |
| title_full | Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement |
| title_fullStr | Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement |
| title_full_unstemmed | Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement |
| title_short | Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement |
| title_sort | learning domain generalized remote sensing image segmentation by multiscale instance disentanglement |
| topic | Domain generalization (DG) instance disentanglement multiscale representation remote sensing image segmentation |
| url | https://ieeexplore.ieee.org/document/10999045/ |
| work_keys_str_mv | AT jieluo learningdomaingeneralizedremotesensingimagesegmentationbymultiscaleinstancedisentanglement AT tianwenluo learningdomaingeneralizedremotesensingimagesegmentationbymultiscaleinstancedisentanglement AT maoyangwang learningdomaingeneralizedremotesensingimagesegmentationbymultiscaleinstancedisentanglement AT linyili learningdomaingeneralizedremotesensingimagesegmentationbymultiscaleinstancedisentanglement AT wenzhang learningdomaingeneralizedremotesensingimagesegmentationbymultiscaleinstancedisentanglement AT lingkuimeng learningdomaingeneralizedremotesensingimagesegmentationbymultiscaleinstancedisentanglement |