Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution
Remarkable strides have been made in super-resolution methods based on deep learning for hyperspectral images (HSIs), which are capable of enhancing the spatial resolution. However, these methods predominantly focus on capturing local features using convolutional neural networks (CNNs), neglecting t...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10980410/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850128648035631104 |
|---|---|
| author | Jiayang Zhang Hongjia Qu Junhao Jia Yaowei Li Bo Jiang Xiaoxuan Chen Jinye Peng |
| author_facet | Jiayang Zhang Hongjia Qu Junhao Jia Yaowei Li Bo Jiang Xiaoxuan Chen Jinye Peng |
| author_sort | Jiayang Zhang |
| collection | DOAJ |
| description | Remarkable strides have been made in super-resolution methods based on deep learning for hyperspectral images (HSIs), which are capable of enhancing the spatial resolution. However, these methods predominantly focus on capturing local features using convolutional neural networks (CNNs), neglecting the comprehensive utilization of global spatial-spectral information. To address this limitation, we innovatively propose a multiscale spatial-spectral CNN-transformer network for hyperspectral image super resolution, namely, MSHSR. MSHSR not only applies the local spatial-spectral characteristics but also innovatively facilitates the collaborative exploration and application of spatial details and spectral data globally. Specifically, we first design a multiscale spatial-spectral fusion module, which integrates dilated-convolution parallel branches and a hybrid spectral attention mechanism to address the strong local correlations in HSIs, effectively capturing and fusing multiscale local spatial-spectral information. Furthermore, in order to fully exploit the global contextual consistency in HSIs, we introduce a sparse spectral transformer module. This module processes the previously obtained local spatial-spectral features, thoroughly exploring the elaborate global interrelationship and long-range dependencies among different spectral bands through a coarse-to-fine strategy. Extensive experimental results on three hyperspectral datasets demonstrate the superior performance of our method, outperforming comparison methods both in quantitative metrics and visual performance. |
| format | Article |
| id | doaj-art-e2095064638742e8b4176041699a7935 |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-e2095064638742e8b4176041699a79352025-08-20T02:33:13ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118121161213210.1109/JSTARS.2025.356584010980410Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-ResolutionJiayang Zhang0Hongjia Qu1Junhao Jia2Yaowei Li3https://orcid.org/0000-0002-9271-2974Bo Jiang4https://orcid.org/0000-0002-4066-1802Xiaoxuan Chen5https://orcid.org/0000-0002-2029-2448Jinye Peng6https://orcid.org/0000-0002-6725-1387School of Information Science and Technology, Northwest University, Xi'an, ChinaSchool of Information Science and Technology, Northwest University, Xi'an, ChinaSchool of Information Science and Technology, Northwest University, Xi'an, ChinaSchool of Information Science and Technology, Northwest University, Xi'an, ChinaSchool of Information Science and Technology, Northwest University, Xi'an, ChinaSchool of Information Science and Technology, Northwest University, Xi'an, ChinaSchool of Information Science and Technology, Northwest University, Xi'an, ChinaRemarkable strides have been made in super-resolution methods based on deep learning for hyperspectral images (HSIs), which are capable of enhancing the spatial resolution. However, these methods predominantly focus on capturing local features using convolutional neural networks (CNNs), neglecting the comprehensive utilization of global spatial-spectral information. To address this limitation, we innovatively propose a multiscale spatial-spectral CNN-transformer network for hyperspectral image super resolution, namely, MSHSR. MSHSR not only applies the local spatial-spectral characteristics but also innovatively facilitates the collaborative exploration and application of spatial details and spectral data globally. Specifically, we first design a multiscale spatial-spectral fusion module, which integrates dilated-convolution parallel branches and a hybrid spectral attention mechanism to address the strong local correlations in HSIs, effectively capturing and fusing multiscale local spatial-spectral information. Furthermore, in order to fully exploit the global contextual consistency in HSIs, we introduce a sparse spectral transformer module. This module processes the previously obtained local spatial-spectral features, thoroughly exploring the elaborate global interrelationship and long-range dependencies among different spectral bands through a coarse-to-fine strategy. Extensive experimental results on three hyperspectral datasets demonstrate the superior performance of our method, outperforming comparison methods both in quantitative metrics and visual performance.https://ieeexplore.ieee.org/document/10980410/Hyperspectral image (HSI)self-attentionsuper-resolution (SR)transformer |
| spellingShingle | Jiayang Zhang Hongjia Qu Junhao Jia Yaowei Li Bo Jiang Xiaoxuan Chen Jinye Peng Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Hyperspectral image (HSI) self-attention super-resolution (SR) transformer |
| title | Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution |
| title_full | Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution |
| title_fullStr | Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution |
| title_full_unstemmed | Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution |
| title_short | Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution |
| title_sort | multiscale spatial spectral cnn transformer network for hyperspectral image super resolution |
| topic | Hyperspectral image (HSI) self-attention super-resolution (SR) transformer |
| url | https://ieeexplore.ieee.org/document/10980410/ |
| work_keys_str_mv | AT jiayangzhang multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution AT hongjiaqu multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution AT junhaojia multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution AT yaoweili multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution AT bojiang multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution AT xiaoxuanchen multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution AT jinyepeng multiscalespatialspectralcnntransformernetworkforhyperspectralimagesuperresolution |