Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder
In this study, we propose a one-branch IncepTAE network to extract local and global hybrid temporal attention simultaneously and congruously for fine-grained satellite image time series (SITS) classification. Transformer and the temporal self-attention mechanism have been the research focus of SITS...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/23/4579 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850261176769839104 |
|---|---|
| author | Zheng Zhang Weixiong Zhang Yu Meng Zhitao Zhao Ping Tang Hongyi Li |
| author_facet | Zheng Zhang Weixiong Zhang Yu Meng Zhitao Zhao Ping Tang Hongyi Li |
| author_sort | Zheng Zhang |
| collection | DOAJ |
| description | In this study, we propose a one-branch IncepTAE network to extract local and global hybrid temporal attention simultaneously and congruously for fine-grained satellite image time series (SITS) classification. Transformer and the temporal self-attention mechanism have been the research focus of SITS classification in recent years. However, its effectiveness seems to diminish in the scenario of fine-grained classification among similar categories, for example, different crop types. Theoretically, most of the existing methods focus on only one type of temporal attention, either global attention or local attention, but actually, both of them are required to achieve fine-grained classification. Even though some works adopt two-branch architecture to extract hybrid attention, they usually lack congruity between different types of temporal attention and hinder the expected discriminating ability. Compared with the existing methods, IncepTAE exhibits multiple methodological novelties. Firstly, we insert average/maximum pooling layers into the calculation of multi-head attention to extract hybrid temporal attention. Secondly, IncepTAE adopts one-branch architecture, which reinforces the interaction and congruity of different temporal information. Thirdly, the proposed IncepTAE is more lightweight due to the use of group convolutions. IncepTAE achieves 95.65% and 97.84% overall accuracy on two challenging datasets, TimeSen2Crop and Ghana. The comparative results with existing <i>state-of-the-art</i> methods demonstrate that IncepTAE is able to achieve superior classification performance and faster inference speed, which is conducive to the large-area application of SITS classification. |
| format | Article |
| id | doaj-art-e26e6ca0f0594edbb1d9b4fd8fcba03d |
| institution | OA Journals |
| issn | 2072-4292 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-e26e6ca0f0594edbb1d9b4fd8fcba03d2025-08-20T01:55:30ZengMDPI AGRemote Sensing2072-42922024-12-011623457910.3390/rs16234579Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention EncoderZheng Zhang0Weixiong Zhang1Yu Meng2Zhitao Zhao3Ping Tang4Hongyi Li5Aerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, ChinaAerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, ChinaAerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, ChinaAerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, ChinaAerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, ChinaAerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, ChinaIn this study, we propose a one-branch IncepTAE network to extract local and global hybrid temporal attention simultaneously and congruously for fine-grained satellite image time series (SITS) classification. Transformer and the temporal self-attention mechanism have been the research focus of SITS classification in recent years. However, its effectiveness seems to diminish in the scenario of fine-grained classification among similar categories, for example, different crop types. Theoretically, most of the existing methods focus on only one type of temporal attention, either global attention or local attention, but actually, both of them are required to achieve fine-grained classification. Even though some works adopt two-branch architecture to extract hybrid attention, they usually lack congruity between different types of temporal attention and hinder the expected discriminating ability. Compared with the existing methods, IncepTAE exhibits multiple methodological novelties. Firstly, we insert average/maximum pooling layers into the calculation of multi-head attention to extract hybrid temporal attention. Secondly, IncepTAE adopts one-branch architecture, which reinforces the interaction and congruity of different temporal information. Thirdly, the proposed IncepTAE is more lightweight due to the use of group convolutions. IncepTAE achieves 95.65% and 97.84% overall accuracy on two challenging datasets, TimeSen2Crop and Ghana. The comparative results with existing <i>state-of-the-art</i> methods demonstrate that IncepTAE is able to achieve superior classification performance and faster inference speed, which is conducive to the large-area application of SITS classification.https://www.mdpi.com/2072-4292/16/23/4579self-attention mechanismsatellite image time seriesclassificationinception nethybrid attentiontemporal attention encoder |
| spellingShingle | Zheng Zhang Weixiong Zhang Yu Meng Zhitao Zhao Ping Tang Hongyi Li Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder Remote Sensing self-attention mechanism satellite image time series classification inception net hybrid attention temporal attention encoder |
| title | Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder |
| title_full | Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder |
| title_fullStr | Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder |
| title_full_unstemmed | Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder |
| title_short | Satellite Image Time-Series Classification with Inception-Enhanced Temporal Attention Encoder |
| title_sort | satellite image time series classification with inception enhanced temporal attention encoder |
| topic | self-attention mechanism satellite image time series classification inception net hybrid attention temporal attention encoder |
| url | https://www.mdpi.com/2072-4292/16/23/4579 |
| work_keys_str_mv | AT zhengzhang satelliteimagetimeseriesclassificationwithinceptionenhancedtemporalattentionencoder AT weixiongzhang satelliteimagetimeseriesclassificationwithinceptionenhancedtemporalattentionencoder AT yumeng satelliteimagetimeseriesclassificationwithinceptionenhancedtemporalattentionencoder AT zhitaozhao satelliteimagetimeseriesclassificationwithinceptionenhancedtemporalattentionencoder AT pingtang satelliteimagetimeseriesclassificationwithinceptionenhancedtemporalattentionencoder AT hongyili satelliteimagetimeseriesclassificationwithinceptionenhancedtemporalattentionencoder |