Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution
The superresolution (SR) technique is a leading solution for achieving high spatial–spectral resolution in hyperspectral (HS) images, which current sensors struggle to provide due to cost and physical constraints. This study presents a multistage optimization framework that leverages high...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11066299/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849714400281231360 |
|---|---|
| author | Zeinab Dehghan Jingxiang Yang Abdolraheem Khader Jian Fang Liang Xiao |
| author_facet | Zeinab Dehghan Jingxiang Yang Abdolraheem Khader Jian Fang Liang Xiao |
| author_sort | Zeinab Dehghan |
| collection | DOAJ |
| description | The superresolution (SR) technique is a leading solution for achieving high spatial–spectral resolution in hyperspectral (HS) images, which current sensors struggle to provide due to cost and physical constraints. This study presents a multistage optimization framework that leverages high- and low-frequency components, along with a quadratic splitting method, to address the SR problem. Traditional model-based approaches often use shallow architectures with limited generalization. To overcome this, we integrated our model into a deep convolutional neural network enhanced by a Transformer module for regularization. Although the Transformer’s capabilities are noteworthy, it can improve in capturing local self-similarity and spectral correlations. Furthermore, these models frequently overlook the importance of multiscale and short-range information. Therefore, we introduce a multiscale architecture that allows Transformers to better capture short- and long-range dependencies. By implementing multiscale spatial-aware and multidepth channel-aware modules, we generate comprehensive deep spatial–spectral prior feature maps. The spatial branch focuses on using local–global prior features for HS image reconstruction, while the spectral branch emphasizes the most informative channels and their correlations. Experiments demonstrate that our method significantly outperforms state-of-the-art fusion-based SR techniques in terms of efficiency. |
| format | Article |
| id | doaj-art-ea28da419bf14c879588cadc27f6cecc |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-ea28da419bf14c879588cadc27f6cecc2025-08-20T03:13:43ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118179261794310.1109/JSTARS.2025.358575111066299Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image SuperresolutionZeinab Dehghan0https://orcid.org/0009-0001-5656-5321Jingxiang Yang1https://orcid.org/0000-0002-1234-0614Abdolraheem Khader2https://orcid.org/0000-0002-1164-3103Jian Fang3https://orcid.org/0000-0002-0781-0183Liang Xiao4https://orcid.org/0000-0003-0178-9384School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaThe superresolution (SR) technique is a leading solution for achieving high spatial–spectral resolution in hyperspectral (HS) images, which current sensors struggle to provide due to cost and physical constraints. This study presents a multistage optimization framework that leverages high- and low-frequency components, along with a quadratic splitting method, to address the SR problem. Traditional model-based approaches often use shallow architectures with limited generalization. To overcome this, we integrated our model into a deep convolutional neural network enhanced by a Transformer module for regularization. Although the Transformer’s capabilities are noteworthy, it can improve in capturing local self-similarity and spectral correlations. Furthermore, these models frequently overlook the importance of multiscale and short-range information. Therefore, we introduce a multiscale architecture that allows Transformers to better capture short- and long-range dependencies. By implementing multiscale spatial-aware and multidepth channel-aware modules, we generate comprehensive deep spatial–spectral prior feature maps. The spatial branch focuses on using local–global prior features for HS image reconstruction, while the spectral branch emphasizes the most informative channels and their correlations. Experiments demonstrate that our method significantly outperforms state-of-the-art fusion-based SR techniques in terms of efficiency.https://ieeexplore.ieee.org/document/11066299/Hyperspectral (HS)image fusionmultispectral (MS)neural networkpanchromatic (PAN)pansharpening |
| spellingShingle | Zeinab Dehghan Jingxiang Yang Abdolraheem Khader Jian Fang Liang Xiao Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Hyperspectral (HS) image fusion multispectral (MS) neural network panchromatic (PAN) pansharpening |
| title | Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution |
| title_full | Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution |
| title_fullStr | Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution |
| title_full_unstemmed | Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution |
| title_short | Learning Spectral–Spatial-Former Deep Prior for Hyperspectral Image Superresolution |
| title_sort | learning spectral x2013 spatial former deep prior for hyperspectral image superresolution |
| topic | Hyperspectral (HS) image fusion multispectral (MS) neural network panchromatic (PAN) pansharpening |
| url | https://ieeexplore.ieee.org/document/11066299/ |
| work_keys_str_mv | AT zeinabdehghan learningspectralx2013spatialformerdeeppriorforhyperspectralimagesuperresolution AT jingxiangyang learningspectralx2013spatialformerdeeppriorforhyperspectralimagesuperresolution AT abdolraheemkhader learningspectralx2013spatialformerdeeppriorforhyperspectralimagesuperresolution AT jianfang learningspectralx2013spatialformerdeeppriorforhyperspectralimagesuperresolution AT liangxiao learningspectralx2013spatialformerdeeppriorforhyperspectralimagesuperresolution |