A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism
Spatial-temporal fusion algorithms commonly encounter difficulties in effectively striking a balance between the extraction of intricate spatial details and changes over time. To mitigate these problems, we propose a spatiotemporal fusion network for remote sensing based on a global context (GC) att...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10766929/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850164947240091648 |
|---|---|
| author | Weisheng Li Yusha Liu Yidong Peng Fengyan Wu |
| author_facet | Weisheng Li Yusha Liu Yidong Peng Fengyan Wu |
| author_sort | Weisheng Li |
| collection | DOAJ |
| description | Spatial-temporal fusion algorithms commonly encounter difficulties in effectively striking a balance between the extraction of intricate spatial details and changes over time. To mitigate these problems, we propose a spatiotemporal fusion network for remote sensing based on a global context (GC) attention mechanism. This network comprises an extracting feature network and a difference network. In the difference network, we introduce a GC attention mechanism that focuses on crucial details across various features, assuming different roles in different modules. A GC attention-based U-block (GCAU) employs a U-structured design and integrates GC attention mechanisms into every layer of its architecture. This enables the module to effectively process regions with pronounced spatial heterogeneity and to adeptly capture spatial differential information. A GC layer (GCL) block comprises five interconnected GC attention blocks interspersed with local residuals. During the training process, these residuals assist in rectifying missing data and bolstering feature transfer, thereby enabling the module to more effectively capture temporally dynamic changes in rapidly evolving areas. This allows the module to better capture time-based changes in fast-changing areas. Supervision of both the final output and intermediate difference images is facilitated by a composite loss function, which improves the fusion quality in the temporal, spatial, and visual domains. The model's robustness and superiority are validated through experimental testing on three datasets, accompanied by subjective and objective evaluations, as well as ablation experiments. |
| format | Article |
| id | doaj-art-c4d68aed897f4aefb59865dab17f0bd2 |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-c4d68aed897f4aefb59865dab17f0bd22025-08-20T02:21:51ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01181451147110.1109/JSTARS.2024.350587410766929A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention MechanismWeisheng Li0https://orcid.org/0000-0002-9033-8245Yusha Liu1https://orcid.org/0009-0007-1894-3989Yidong Peng2https://orcid.org/0000-0003-3779-0360Fengyan Wu3Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, ChinaChongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, ChinaChongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, ChinaChongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, ChinaSpatial-temporal fusion algorithms commonly encounter difficulties in effectively striking a balance between the extraction of intricate spatial details and changes over time. To mitigate these problems, we propose a spatiotemporal fusion network for remote sensing based on a global context (GC) attention mechanism. This network comprises an extracting feature network and a difference network. In the difference network, we introduce a GC attention mechanism that focuses on crucial details across various features, assuming different roles in different modules. A GC attention-based U-block (GCAU) employs a U-structured design and integrates GC attention mechanisms into every layer of its architecture. This enables the module to effectively process regions with pronounced spatial heterogeneity and to adeptly capture spatial differential information. A GC layer (GCL) block comprises five interconnected GC attention blocks interspersed with local residuals. During the training process, these residuals assist in rectifying missing data and bolstering feature transfer, thereby enabling the module to more effectively capture temporally dynamic changes in rapidly evolving areas. This allows the module to better capture time-based changes in fast-changing areas. Supervision of both the final output and intermediate difference images is facilitated by a composite loss function, which improves the fusion quality in the temporal, spatial, and visual domains. The model's robustness and superiority are validated through experimental testing on three datasets, accompanied by subjective and objective evaluations, as well as ablation experiments.https://ieeexplore.ieee.org/document/10766929/Global context (GC)global context attention-based U-block (GCAU)global context layer (GCL)remote sensingspatiotemporal fusion |
| spellingShingle | Weisheng Li Yusha Liu Yidong Peng Fengyan Wu A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Global context (GC) global context attention-based U-block (GCAU) global context layer (GCL) remote sensing spatiotemporal fusion |
| title | A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism |
| title_full | A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism |
| title_fullStr | A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism |
| title_full_unstemmed | A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism |
| title_short | A Spatiotemporal Fusion Network for Remote Sensing Based on Global Context Attention Mechanism |
| title_sort | spatiotemporal fusion network for remote sensing based on global context attention mechanism |
| topic | Global context (GC) global context attention-based U-block (GCAU) global context layer (GCL) remote sensing spatiotemporal fusion |
| url | https://ieeexplore.ieee.org/document/10766929/ |
| work_keys_str_mv | AT weishengli aspatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT yushaliu aspatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT yidongpeng aspatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT fengyanwu aspatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT weishengli spatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT yushaliu spatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT yidongpeng spatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism AT fengyanwu spatiotemporalfusionnetworkforremotesensingbasedonglobalcontextattentionmechanism |