Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network
Hyperspectral and multispectral remote sensing image fusion is an optimal approach for generating hyperspectral–spatial-resolution images, effectively overcoming the physical limitations of sensors. In transformer-based image fusion methods constrained by the local window self-attention mechanism, t...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/17/12/1973 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849425332359135232 |
|---|---|
| author | Yilin He Heng Li Miaosen Zhang Shuangqi Liu Chunyu Zhu Bingxia Xin Jun Wang Qiong Wu |
| author_facet | Yilin He Heng Li Miaosen Zhang Shuangqi Liu Chunyu Zhu Bingxia Xin Jun Wang Qiong Wu |
| author_sort | Yilin He |
| collection | DOAJ |
| description | Hyperspectral and multispectral remote sensing image fusion is an optimal approach for generating hyperspectral–spatial-resolution images, effectively overcoming the physical limitations of sensors. In transformer-based image fusion methods constrained by the local window self-attention mechanism, the extraction of global information and coordinated contextual features is often insufficient. Fusion that aims to emphasize spatial–spectral heterogeneous characteristics may significantly enhance the robustness of joint representation for multi-source data. To address these issues, this study proposes a hyperspectral and multispectral remote sensing image fusion method based on a retractable spatial–spectral transformer network (RSST) and introduces the attention retractable mechanism into the field of remote sensing image fusion. Furthermore, a gradient spatial–spectral recovery block is incorporated to effectively mitigate the limitations of token interactions and the loss of spatial–spectral edge information. A series of experiments across multiple scales demonstrate that RSST exhibits significant advantages over existing mainstream image fusion algorithms. |
| format | Article |
| id | doaj-art-ad0f7804f9bb4ac792eff9c4bc993d22 |
| institution | Kabale University |
| issn | 2072-4292 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-ad0f7804f9bb4ac792eff9c4bc993d222025-08-20T03:29:48ZengMDPI AGRemote Sensing2072-42922025-06-011712197310.3390/rs17121973Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer NetworkYilin He0Heng Li1Miaosen Zhang2Shuangqi Liu3Chunyu Zhu4Bingxia Xin5Jun Wang6Qiong Wu7College of Geo-Exploration Science and Technology, Jilin University, No. 938 Ximinzhu Street, Changchun 130026, ChinaCollege of Software, Jilin University, No. 2699 Qianjin Street, Changchun 130012, ChinaCollege of Geo-Exploration Science and Technology, Jilin University, No. 938 Ximinzhu Street, Changchun 130026, ChinaCollege of Geo-Exploration Science and Technology, Jilin University, No. 938 Ximinzhu Street, Changchun 130026, ChinaHangzhou Institute of Technology, Xidian University, No. 177 Lvxue Road, Hangzhou 311231, ChinaCollege of Geo-Exploration Science and Technology, Jilin University, No. 938 Ximinzhu Street, Changchun 130026, ChinaCollege of Geo-Exploration Science and Technology, Jilin University, No. 938 Ximinzhu Street, Changchun 130026, ChinaCollege of Geo-Exploration Science and Technology, Jilin University, No. 938 Ximinzhu Street, Changchun 130026, ChinaHyperspectral and multispectral remote sensing image fusion is an optimal approach for generating hyperspectral–spatial-resolution images, effectively overcoming the physical limitations of sensors. In transformer-based image fusion methods constrained by the local window self-attention mechanism, the extraction of global information and coordinated contextual features is often insufficient. Fusion that aims to emphasize spatial–spectral heterogeneous characteristics may significantly enhance the robustness of joint representation for multi-source data. To address these issues, this study proposes a hyperspectral and multispectral remote sensing image fusion method based on a retractable spatial–spectral transformer network (RSST) and introduces the attention retractable mechanism into the field of remote sensing image fusion. Furthermore, a gradient spatial–spectral recovery block is incorporated to effectively mitigate the limitations of token interactions and the loss of spatial–spectral edge information. A series of experiments across multiple scales demonstrate that RSST exhibits significant advantages over existing mainstream image fusion algorithms.https://www.mdpi.com/2072-4292/17/12/1973image fusionhyperspectral imagemultispectral imagetransformerretractable attentiongradient spatial–spectral recovery |
| spellingShingle | Yilin He Heng Li Miaosen Zhang Shuangqi Liu Chunyu Zhu Bingxia Xin Jun Wang Qiong Wu Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network Remote Sensing image fusion hyperspectral image multispectral image transformer retractable attention gradient spatial–spectral recovery |
| title | Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network |
| title_full | Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network |
| title_fullStr | Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network |
| title_full_unstemmed | Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network |
| title_short | Hyperspectral and Multispectral Remote Sensing Image Fusion Based on a Retractable Spatial–Spectral Transformer Network |
| title_sort | hyperspectral and multispectral remote sensing image fusion based on a retractable spatial spectral transformer network |
| topic | image fusion hyperspectral image multispectral image transformer retractable attention gradient spatial–spectral recovery |
| url | https://www.mdpi.com/2072-4292/17/12/1973 |
| work_keys_str_mv | AT yilinhe hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT hengli hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT miaosenzhang hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT shuangqiliu hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT chunyuzhu hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT bingxiaxin hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT junwang hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork AT qiongwu hyperspectralandmultispectralremotesensingimagefusionbasedonaretractablespatialspectraltransformernetwork |