Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening
Pansharpening is the process of fusing a multispectral (MS) image with a panchromatic image to produce a high-resolution MS (HRMS) image. However, existing techniques face challenges in integrating long-range dependencies to correct locally misaligned features, which results in spatial-spectral dist...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10845120/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849716161487306752 |
|---|---|
| author | Qun Song Hangyuan Lu Chang Xu Rixian Liu Weiguo Wan Wei Tu |
| author_facet | Qun Song Hangyuan Lu Chang Xu Rixian Liu Weiguo Wan Wei Tu |
| author_sort | Qun Song |
| collection | DOAJ |
| description | Pansharpening is the process of fusing a multispectral (MS) image with a panchromatic image to produce a high-resolution MS (HRMS) image. However, existing techniques face challenges in integrating long-range dependencies to correct locally misaligned features, which results in spatial-spectral distortions. Moreover, these methods tend to be computationally expensive. To address these challenges, we propose a novel detail injection algorithm and develop the invertible attention-guided adaptive convolution and dual-domain Transformer (IACDT) network. In IACDT, we designed an invertible attention mechanism embedded with spectral-spatial attention to efficiently and losslessly extract locally spatial-spectral-aware detail information. In addition, we presented a frequency-spatial dual-domain attention mechanism that combines a frequency-enhanced Transformer and a spatial window Transformer for long-range contextual detail feature correction. This architecture effectively integrates local detail features with long-range dependencies, enabling the model to correct both local misalignments and global inconsistencies. The final HRMS image is obtained through a reconstruction block that consists of residual multireceptive field attention. Extensive experiments demonstrate that IACDT achieves superior fusion performance, computational efficiency, and outstanding results in downstream tasks compared to state-of-the-art methods. |
| format | Article |
| id | doaj-art-a3eaaaa1abf8470fbd66a443fda80f1a |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-a3eaaaa1abf8470fbd66a443fda80f1a2025-08-20T03:13:07ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01185217523110.1109/JSTARS.2025.353135310845120Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for PansharpeningQun Song0Hangyuan Lu1https://orcid.org/0000-0002-2829-8553Chang Xu2https://orcid.org/0009-0008-3253-4839Rixian Liu3https://orcid.org/0000-0002-6217-997XWeiguo Wan4https://orcid.org/0000-0002-3537-979XWei Tu5https://orcid.org/0000-0002-2673-2192College of Information Engineering, Jinhua University of Vocational Technology, Jinhua, ChinaCollege of Information Engineering, Jinhua University of Vocational Technology, Jinhua, ChinaHangzhou Consumer Council, Hangzhou, ChinaCollege of Information Engineering, Jinhua University of Vocational Technology, Jinhua, ChinaSchool of Software and Internet of Things Engineering, Jiangxi University of Finance and Economics, Nanchang, ChinaSchool of Big Data Science, Jiangxi Science and Technology Normal University, Nanchang, ChinaPansharpening is the process of fusing a multispectral (MS) image with a panchromatic image to produce a high-resolution MS (HRMS) image. However, existing techniques face challenges in integrating long-range dependencies to correct locally misaligned features, which results in spatial-spectral distortions. Moreover, these methods tend to be computationally expensive. To address these challenges, we propose a novel detail injection algorithm and develop the invertible attention-guided adaptive convolution and dual-domain Transformer (IACDT) network. In IACDT, we designed an invertible attention mechanism embedded with spectral-spatial attention to efficiently and losslessly extract locally spatial-spectral-aware detail information. In addition, we presented a frequency-spatial dual-domain attention mechanism that combines a frequency-enhanced Transformer and a spatial window Transformer for long-range contextual detail feature correction. This architecture effectively integrates local detail features with long-range dependencies, enabling the model to correct both local misalignments and global inconsistencies. The final HRMS image is obtained through a reconstruction block that consists of residual multireceptive field attention. Extensive experiments demonstrate that IACDT achieves superior fusion performance, computational efficiency, and outstanding results in downstream tasks compared to state-of-the-art methods.https://ieeexplore.ieee.org/document/10845120/Adaptive convolutiondual-domainpansharpeningtransformer |
| spellingShingle | Qun Song Hangyuan Lu Chang Xu Rixian Liu Weiguo Wan Wei Tu Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Adaptive convolution dual-domain pansharpening transformer |
| title | Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening |
| title_full | Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening |
| title_fullStr | Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening |
| title_full_unstemmed | Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening |
| title_short | Invertible Attention-Guided Adaptive Convolution and Dual-Domain Transformer for Pansharpening |
| title_sort | invertible attention guided adaptive convolution and dual domain transformer for pansharpening |
| topic | Adaptive convolution dual-domain pansharpening transformer |
| url | https://ieeexplore.ieee.org/document/10845120/ |
| work_keys_str_mv | AT qunsong invertibleattentionguidedadaptiveconvolutionanddualdomaintransformerforpansharpening AT hangyuanlu invertibleattentionguidedadaptiveconvolutionanddualdomaintransformerforpansharpening AT changxu invertibleattentionguidedadaptiveconvolutionanddualdomaintransformerforpansharpening AT rixianliu invertibleattentionguidedadaptiveconvolutionanddualdomaintransformerforpansharpening AT weiguowan invertibleattentionguidedadaptiveconvolutionanddualdomaintransformerforpansharpening AT weitu invertibleattentionguidedadaptiveconvolutionanddualdomaintransformerforpansharpening |