A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening
Over the past decades, pansharpening technologies have received much attention due to the spatial detail enhancement they introduce into multispectral (MS) images by referencing them to panchromatic images (PAN). Traditional approaches struggle to capture nonlinear relationships between the original...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11075521/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849404397004521472 |
|---|---|
| author | Hailiang Lu Mercedes E. Paoletti Juan M. Haut Sergio Moreno-Alvarez Guangsheng Chen Weipeng Jing |
| author_facet | Hailiang Lu Mercedes E. Paoletti Juan M. Haut Sergio Moreno-Alvarez Guangsheng Chen Weipeng Jing |
| author_sort | Hailiang Lu |
| collection | DOAJ |
| description | Over the past decades, pansharpening technologies have received much attention due to the spatial detail enhancement they introduce into multispectral (MS) images by referencing them to panchromatic images (PAN). Traditional approaches struggle to capture nonlinear relationships between the original MS-PAN pair and the pansharpened MS, whilst deep learning (DL)-based methods introduce new challenges. Unsupervised pansharpening is prone to distortion due to the unavailability of reference images and errors in modeling the degradation process. Supervised methods are trained at reduced resolutions before migrating to full resolution, which lead to undesirable results due to scale variations. Meanwhile, the constructed training data cannot encompass all possible scenes, hindering the reconstruction capability of unknown scenes. To tackle these challenges, we propose a novel zero-shot Pansharpening Network (<monospace>ZSPNet</monospace>), which exclusively conducts training and testing on the target image pair, ensuring the robust performance in unknown scenes. Furthermore, <monospace>ZSPNet</monospace> method can effectively reduce the scale variance due to its tailored network using an appropriate patch size. As a result, by integrating 3-D convolutional neural networks (3DCNN), spatial attention and channel attention, <monospace>ZSPNet</monospace> is capable of accurately reconstructing MS with enhanced spatial resolution. The experiments were conducted on the public dataset PAirMax, which have challenging scenes captured by different sensors. Compared to some state-of-the-art traditional and DL-based methods, <monospace>ZSPNet</monospace> demonstrates superior performance in both quantitative assessments and visual results. |
| format | Article |
| id | doaj-art-65c86b166c3c46f693f4b8d07a899712 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-65c86b166c3c46f693f4b8d07a8997122025-08-20T03:37:01ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118198051982010.1109/JSTARS.2025.358724411075521A Novel Spectral-Spatial Attention Network for Zero-Shot PansharpeningHailiang Lu0https://orcid.org/0000-0003-0094-184XMercedes E. Paoletti1https://orcid.org/0000-0003-1030-3729Juan M. Haut2https://orcid.org/0000-0001-6701-961XSergio Moreno-Alvarez3https://orcid.org/0000-0002-1858-9920Guangsheng Chen4https://orcid.org/0009-0006-8983-9698Weipeng Jing5https://orcid.org/0000-0001-7933-6946College of Computer and Control Engineering, Northeast Forestry University, Harbin, ChinaDepartment of Technology of Computers and Communications, University of Extremadura, Cáceres, SpainDepartment of Technology of Computers and Communications, University of Extremadura, Cáceres, SpainDepartment of Computer Systems and Languages, National University of Distance Education, Madrid, SpainCollege of Computer and Control Engineering, Northeast Forestry University, Harbin, ChinaCollege of Computer and Control Engineering, Northeast Forestry University, Harbin, ChinaOver the past decades, pansharpening technologies have received much attention due to the spatial detail enhancement they introduce into multispectral (MS) images by referencing them to panchromatic images (PAN). Traditional approaches struggle to capture nonlinear relationships between the original MS-PAN pair and the pansharpened MS, whilst deep learning (DL)-based methods introduce new challenges. Unsupervised pansharpening is prone to distortion due to the unavailability of reference images and errors in modeling the degradation process. Supervised methods are trained at reduced resolutions before migrating to full resolution, which lead to undesirable results due to scale variations. Meanwhile, the constructed training data cannot encompass all possible scenes, hindering the reconstruction capability of unknown scenes. To tackle these challenges, we propose a novel zero-shot Pansharpening Network (<monospace>ZSPNet</monospace>), which exclusively conducts training and testing on the target image pair, ensuring the robust performance in unknown scenes. Furthermore, <monospace>ZSPNet</monospace> method can effectively reduce the scale variance due to its tailored network using an appropriate patch size. As a result, by integrating 3-D convolutional neural networks (3DCNN), spatial attention and channel attention, <monospace>ZSPNet</monospace> is capable of accurately reconstructing MS with enhanced spatial resolution. The experiments were conducted on the public dataset PAirMax, which have challenging scenes captured by different sensors. Compared to some state-of-the-art traditional and DL-based methods, <monospace>ZSPNet</monospace> demonstrates superior performance in both quantitative assessments and visual results.https://ieeexplore.ieee.org/document/11075521/3DCNNchannel attentionpansharpeningremote sensingspatial attentionzero-shot learning |
| spellingShingle | Hailiang Lu Mercedes E. Paoletti Juan M. Haut Sergio Moreno-Alvarez Guangsheng Chen Weipeng Jing A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 3DCNN channel attention pansharpening remote sensing spatial attention zero-shot learning |
| title | A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening |
| title_full | A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening |
| title_fullStr | A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening |
| title_full_unstemmed | A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening |
| title_short | A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening |
| title_sort | novel spectral spatial attention network for zero shot pansharpening |
| topic | 3DCNN channel attention pansharpening remote sensing spatial attention zero-shot learning |
| url | https://ieeexplore.ieee.org/document/11075521/ |
| work_keys_str_mv | AT hailianglu anovelspectralspatialattentionnetworkforzeroshotpansharpening AT mercedesepaoletti anovelspectralspatialattentionnetworkforzeroshotpansharpening AT juanmhaut anovelspectralspatialattentionnetworkforzeroshotpansharpening AT sergiomorenoalvarez anovelspectralspatialattentionnetworkforzeroshotpansharpening AT guangshengchen anovelspectralspatialattentionnetworkforzeroshotpansharpening AT weipengjing anovelspectralspatialattentionnetworkforzeroshotpansharpening AT hailianglu novelspectralspatialattentionnetworkforzeroshotpansharpening AT mercedesepaoletti novelspectralspatialattentionnetworkforzeroshotpansharpening AT juanmhaut novelspectralspatialattentionnetworkforzeroshotpansharpening AT sergiomorenoalvarez novelspectralspatialattentionnetworkforzeroshotpansharpening AT guangshengchen novelspectralspatialattentionnetworkforzeroshotpansharpening AT weipengjing novelspectralspatialattentionnetworkforzeroshotpansharpening |