Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network
Recently, multilayer perceptron (MLPs)-based methods in computer vision have attracted much attention due to the ability of learning long-range dependencies. However, MLPs-based methods usually treat all the tokens equally, which is difficult to segment challenging cloud regions. In this article, we...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11098898/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849762555144175616 |
|---|---|
| author | Shuang Liu Jiafeng Zhang Zhong Zhang Shuzhen Hu Baihua Xiao |
| author_facet | Shuang Liu Jiafeng Zhang Zhong Zhang Shuzhen Hu Baihua Xiao |
| author_sort | Shuang Liu |
| collection | DOAJ |
| description | Recently, multilayer perceptron (MLPs)-based methods in computer vision have attracted much attention due to the ability of learning long-range dependencies. However, MLPs-based methods usually treat all the tokens equally, which is difficult to segment challenging cloud regions. In this article, we propose a novel network named convolution-MLP network (Con-MLPNet) for ground-based remote sensing cloud image segmentation, which could effectively learn long-range dependencies via the combination of MLPs and the attention mechanism. To this end, we propose the attention-guided MLPs module to highlight salient features and suppress irrelevant features from the spatial and channel aspects. Meanwhile, different from existing MLPs methods where the long-range dependencies are learned from one single scale, we propose the dilated MLPs (DMLPs) to learn long-range dependencies at different scales by sampling different channels of tokens. Furthermore, we design the parallel dilated MLPs module to integrate multiple DMLPs with different parameters in order to extract multiscale information. We conduct a series of experiments on three public ground-based cloud image segmentation datasets, i.e., TLCDD, SWIMSEG, and TCDD, and the results demonstrate that the proposed Con-MLPNet achieves state-of-the-art performance. Specifically, on the TLCDD dataset, our method surpasses the competing method across all five evaluation metrics, with the improvements of 3.3% in precision, 2.48% in recall, 3.74% in F-score, 1.76% in accuracy, and 4.0% in IoU over the second-best results. |
| format | Article |
| id | doaj-art-7feaba50c3b642f4b7ee0476dcefde75 |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-7feaba50c3b642f4b7ee0476dcefde752025-08-20T03:05:42ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118195631957610.1109/JSTARS.2025.359347311098898Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP NetworkShuang Liu0https://orcid.org/0000-0002-9027-0690Jiafeng Zhang1https://orcid.org/0009-0004-9363-8800Zhong Zhang2https://orcid.org/0000-0002-2993-8612Shuzhen Hu3Baihua Xiao4https://orcid.org/0000-0003-3941-1141Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, ChinaTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, ChinaTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, ChinaMeteorological Observation Center, China Meteorological Administration, Beijing, ChinaState Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, ChinaRecently, multilayer perceptron (MLPs)-based methods in computer vision have attracted much attention due to the ability of learning long-range dependencies. However, MLPs-based methods usually treat all the tokens equally, which is difficult to segment challenging cloud regions. In this article, we propose a novel network named convolution-MLP network (Con-MLPNet) for ground-based remote sensing cloud image segmentation, which could effectively learn long-range dependencies via the combination of MLPs and the attention mechanism. To this end, we propose the attention-guided MLPs module to highlight salient features and suppress irrelevant features from the spatial and channel aspects. Meanwhile, different from existing MLPs methods where the long-range dependencies are learned from one single scale, we propose the dilated MLPs (DMLPs) to learn long-range dependencies at different scales by sampling different channels of tokens. Furthermore, we design the parallel dilated MLPs module to integrate multiple DMLPs with different parameters in order to extract multiscale information. We conduct a series of experiments on three public ground-based cloud image segmentation datasets, i.e., TLCDD, SWIMSEG, and TCDD, and the results demonstrate that the proposed Con-MLPNet achieves state-of-the-art performance. Specifically, on the TLCDD dataset, our method surpasses the competing method across all five evaluation metrics, with the improvements of 3.3% in precision, 2.48% in recall, 3.74% in F-score, 1.76% in accuracy, and 4.0% in IoU over the second-best results.https://ieeexplore.ieee.org/document/11098898/Attention-guided MLPs (AGMs)parallel dilated MLPs (PDMs)remote sensing cloud image segmentation |
| spellingShingle | Shuang Liu Jiafeng Zhang Zhong Zhang Shuzhen Hu Baihua Xiao Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Attention-guided MLPs (AGMs) parallel dilated MLPs (PDMs) remote sensing cloud image segmentation |
| title | Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network |
| title_full | Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network |
| title_fullStr | Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network |
| title_full_unstemmed | Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network |
| title_short | Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network |
| title_sort | ground based remote sensing cloud image segmentation using convolution mlp network |
| topic | Attention-guided MLPs (AGMs) parallel dilated MLPs (PDMs) remote sensing cloud image segmentation |
| url | https://ieeexplore.ieee.org/document/11098898/ |
| work_keys_str_mv | AT shuangliu groundbasedremotesensingcloudimagesegmentationusingconvolutionmlpnetwork AT jiafengzhang groundbasedremotesensingcloudimagesegmentationusingconvolutionmlpnetwork AT zhongzhang groundbasedremotesensingcloudimagesegmentationusingconvolutionmlpnetwork AT shuzhenhu groundbasedremotesensingcloudimagesegmentationusingconvolutionmlpnetwork AT baihuaxiao groundbasedremotesensingcloudimagesegmentationusingconvolutionmlpnetwork |