Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image
The tradeoff between spatial and spectral resolution in sensor design is inevitable, and spatial–spectral fusion aims to use low spatial resolution hyperspectral image (HSI) and high spatial resolution (HR) multispectral image (MSI) obtained at the same time and in the same area to recons...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10695805/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850209985375502336 |
|---|---|
| author | Guangwei Zhao Haitao Wu Dexiang Luo Xu Ou Yu Zhang |
| author_facet | Guangwei Zhao Haitao Wu Dexiang Luo Xu Ou Yu Zhang |
| author_sort | Guangwei Zhao |
| collection | DOAJ |
| description | The tradeoff between spatial and spectral resolution in sensor design is inevitable, and spatial–spectral fusion aims to use low spatial resolution hyperspectral image (HSI) and high spatial resolution (HR) multispectral image (MSI) obtained at the same time and in the same area to reconstruct HR HSI. Recently, a large number of deep-learning methods have been applied in this field and achieved success. However, these methods do not fully utilize the characteristics of data for network design, and cannot guarantee effective computational efficiency in extracting local and global features. To solve the above problems, we designed a spatial–spectral interaction super-resolution convolutional neural network (CNN)–Mamba fusion network for satellite HSI and MSI, which uses mutual guidance to improve the spatial and spectral resolution of different data, and obtains the final fused image through feature fusion. In addition, we combined Mamba with CNN to effectively explore global and local features of images. Extensive experiments have proven that our method can reconstruct fused images of high quality and is superior to current state-of-the-art fusion methods. |
| format | Article |
| id | doaj-art-c7ebf0078e2148758e94f94ae826de6e |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-c7ebf0078e2148758e94f94ae826de6e2025-08-20T02:09:52ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352024-01-0117184891850110.1109/JSTARS.2024.346918410695805Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral ImageGuangwei Zhao0https://orcid.org/0009-0008-5863-9639Haitao Wu1Dexiang Luo2https://orcid.org/0009-0002-3810-4289Xu Ou3Yu Zhang4College of Computing and Artificial Intelligence, Huanghuai University, Zhumadian, ChinaCollege of Computing and Artificial Intelligence, Huanghuai University, Zhumadian, ChinaInformation Centre of Guangxi Medical University, Nanning, ChinaInformation Centre of Guangxi Medical University, Nanning, ChinaCollege of Computing and Artificial Intelligence, Huanghuai University, Zhumadian, ChinaThe tradeoff between spatial and spectral resolution in sensor design is inevitable, and spatial–spectral fusion aims to use low spatial resolution hyperspectral image (HSI) and high spatial resolution (HR) multispectral image (MSI) obtained at the same time and in the same area to reconstruct HR HSI. Recently, a large number of deep-learning methods have been applied in this field and achieved success. However, these methods do not fully utilize the characteristics of data for network design, and cannot guarantee effective computational efficiency in extracting local and global features. To solve the above problems, we designed a spatial–spectral interaction super-resolution convolutional neural network (CNN)–Mamba fusion network for satellite HSI and MSI, which uses mutual guidance to improve the spatial and spectral resolution of different data, and obtains the final fused image through feature fusion. In addition, we combined Mamba with CNN to effectively explore global and local features of images. Extensive experiments have proven that our method can reconstruct fused images of high quality and is superior to current state-of-the-art fusion methods.https://ieeexplore.ieee.org/document/10695805/Convolutional neural network (CNN)fusionhyperspectralMambamultispectral |
| spellingShingle | Guangwei Zhao Haitao Wu Dexiang Luo Xu Ou Yu Zhang Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Convolutional neural network (CNN) fusion hyperspectral Mamba multispectral |
| title | Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image |
| title_full | Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image |
| title_fullStr | Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image |
| title_full_unstemmed | Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image |
| title_short | Spatial–Spectral Interaction Super-Resolution CNN–Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image |
| title_sort | spatial x2013 spectral interaction super resolution cnn x2013 mamba network for fusion of satellite hyperspectral and multispectral image |
| topic | Convolutional neural network (CNN) fusion hyperspectral Mamba multispectral |
| url | https://ieeexplore.ieee.org/document/10695805/ |
| work_keys_str_mv | AT guangweizhao spatialx2013spectralinteractionsuperresolutioncnnx2013mambanetworkforfusionofsatellitehyperspectralandmultispectralimage AT haitaowu spatialx2013spectralinteractionsuperresolutioncnnx2013mambanetworkforfusionofsatellitehyperspectralandmultispectralimage AT dexiangluo spatialx2013spectralinteractionsuperresolutioncnnx2013mambanetworkforfusionofsatellitehyperspectralandmultispectralimage AT xuou spatialx2013spectralinteractionsuperresolutioncnnx2013mambanetworkforfusionofsatellitehyperspectralandmultispectralimage AT yuzhang spatialx2013spectralinteractionsuperresolutioncnnx2013mambanetworkforfusionofsatellitehyperspectralandmultispectralimage |