Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use
Land use classification remains a significant challenge in remote sensing semantic segmentation. While convolutional neural networks (CNNs) are widely used, their inherent limitations, such as restricted receptive fields, hinder their widespread application in remote sensing. Additionally, the scarc...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/17/2/290 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832587605589360640 |
|---|---|
| author | Miaomiao Chen Lianfa Li |
| author_facet | Miaomiao Chen Lianfa Li |
| author_sort | Miaomiao Chen |
| collection | DOAJ |
| description | Land use classification remains a significant challenge in remote sensing semantic segmentation. While convolutional neural networks (CNNs) are widely used, their inherent limitations, such as restricted receptive fields, hinder their widespread application in remote sensing. Additionally, the scarcity of labeled remote sensing data and domain shift issues adversely impact deep learning model performance. This study proposes a hierarchical transfer learning framework for fine-category semantic segmentation tasks, leveraging the powerful global relationship modeling capabilities of Transformer models to classify land use in Dongpo District, Meishan City, in mainland China. Our framework represents multilevel transfer learning, progressing from non-remote sensing classification to coarse classification, then to the refined classification of remote sensing. We compared the performance of Transformer models with representative baseline CNNs like U-Net and DeepLab V3+. Results show that the Swin-Unet model outperforms the other models used in this study. It achieved the highest test mean intersection over union (MIoU) of 0.837 and 0.810 for residential and transportation in level 1 (coarse) classification, respectively, and 0.545 for irrigated land in level 2 (fine-grained) classification. Transfer learning from pre-trained models significantly enhanced semantic segmentation accuracy compared to random parameter initialization (ranging from 0.4% to 17.7%), with up to a 17.7% improvement in test MIoU for the public land category. The hierarchical transfer learning framework further improved segmentation accuracy for corresponding level 2 categories, leveraging pre-trained level 1 models. Our study shows the applicability of Transformer-based transfer learning in remote sensing land use classification. |
| format | Article |
| id | doaj-art-e97e91019ea944b3868c87c98a087117 |
| institution | Kabale University |
| issn | 2072-4292 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-e97e91019ea944b3868c87c98a0871172025-01-24T13:48:01ZengMDPI AGRemote Sensing2072-42922025-01-0117229010.3390/rs17020290Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land UseMiaomiao Chen0Lianfa Li1State Key Laboratory of Resources and Environmental Information Systems, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Datun Road, Beijing 100101, ChinaState Key Laboratory of Resources and Environmental Information Systems, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Datun Road, Beijing 100101, ChinaLand use classification remains a significant challenge in remote sensing semantic segmentation. While convolutional neural networks (CNNs) are widely used, their inherent limitations, such as restricted receptive fields, hinder their widespread application in remote sensing. Additionally, the scarcity of labeled remote sensing data and domain shift issues adversely impact deep learning model performance. This study proposes a hierarchical transfer learning framework for fine-category semantic segmentation tasks, leveraging the powerful global relationship modeling capabilities of Transformer models to classify land use in Dongpo District, Meishan City, in mainland China. Our framework represents multilevel transfer learning, progressing from non-remote sensing classification to coarse classification, then to the refined classification of remote sensing. We compared the performance of Transformer models with representative baseline CNNs like U-Net and DeepLab V3+. Results show that the Swin-Unet model outperforms the other models used in this study. It achieved the highest test mean intersection over union (MIoU) of 0.837 and 0.810 for residential and transportation in level 1 (coarse) classification, respectively, and 0.545 for irrigated land in level 2 (fine-grained) classification. Transfer learning from pre-trained models significantly enhanced semantic segmentation accuracy compared to random parameter initialization (ranging from 0.4% to 17.7%), with up to a 17.7% improvement in test MIoU for the public land category. The hierarchical transfer learning framework further improved segmentation accuracy for corresponding level 2 categories, leveraging pre-trained level 1 models. Our study shows the applicability of Transformer-based transfer learning in remote sensing land use classification.https://www.mdpi.com/2072-4292/17/2/290land usesemantic segmentationtransformerhierarchical transfer learning |
| spellingShingle | Miaomiao Chen Lianfa Li Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use Remote Sensing land use semantic segmentation transformer hierarchical transfer learning |
| title | Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use |
| title_full | Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use |
| title_fullStr | Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use |
| title_full_unstemmed | Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use |
| title_short | Hierarchical Transfer Learning with Transformers to Improve Semantic Segmentation in Remote Sensing Land Use |
| title_sort | hierarchical transfer learning with transformers to improve semantic segmentation in remote sensing land use |
| topic | land use semantic segmentation transformer hierarchical transfer learning |
| url | https://www.mdpi.com/2072-4292/17/2/290 |
| work_keys_str_mv | AT miaomiaochen hierarchicaltransferlearningwithtransformerstoimprovesemanticsegmentationinremotesensinglanduse AT lianfali hierarchicaltransferlearningwithtransformerstoimprovesemanticsegmentationinremotesensinglanduse |