HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution
The core challenge of hyperspectral compressive imaging is to reconstruct the three-dimensional hyperspectral image from two-dimensional compressed measurements. While recent deep learning-based methods have demonsetrated outstanding performance, they often lack robust theoretical interpretability....
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11022735/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850122086375227392 |
|---|---|
| author | Le Sun Xihan Ma Xinyu Wang Qiao Chen Zebin Wu |
| author_facet | Le Sun Xihan Ma Xinyu Wang Qiao Chen Zebin Wu |
| author_sort | Le Sun |
| collection | DOAJ |
| description | The core challenge of hyperspectral compressive imaging is to reconstruct the three-dimensional hyperspectral image from two-dimensional compressed measurements. While recent deep learning-based methods have demonsetrated outstanding performance, they often lack robust theoretical interpretability. Conversely, traditional iterative optimization algorithms are built upon sound mathematical derivations. To combine the advantages of both approaches, we propose a spectral transformer network, termed STTODNet, which integrates deep tensor decomposition and omni-dimensional dynamic convolution (ODConv). Specifically, we incorporate a deep Tucker decomposition module within the self-attention mechanism to effectively extract low-rank prior features inherent in the hyperspectral image. Moreover, we replace the conventional linear projection layer with ODConv to substantially improve feature extraction capabilities. A three-scale U-Net network structure is designed as the approximate operator for solving the prior within our deep unfolding network architecture. Extensive experimental results demonstrate that STTODNet achieves superior results in terms of reconstruction quality, interpretability, and computational efficiency when compared to state-of-the-art methods. |
| format | Article |
| id | doaj-art-d829eb9fc87d43209ada7f59f98f153c |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-d829eb9fc87d43209ada7f59f98f153c2025-08-20T02:34:55ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118149881500010.1109/JSTARS.2025.357617911022735HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic ConvolutionLe Sun0https://orcid.org/0000-0001-6465-8678Xihan Ma1Xinyu Wang2Qiao Chen3https://orcid.org/0000-0002-6458-8742Zebin Wu4https://orcid.org/0000-0002-7162-0202School of Computer Science, Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Computer Science, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Computer Science, Nanjing University of Information Science and Technology, Nanjing, ChinaInstitute of Forest Resource Information Techniques, Chinese Academy of Forestry (CAF), Beijing, ChinaSchool of Computer Engineering, Nanjing University of Science and Technology, Nanjing, ChinaThe core challenge of hyperspectral compressive imaging is to reconstruct the three-dimensional hyperspectral image from two-dimensional compressed measurements. While recent deep learning-based methods have demonsetrated outstanding performance, they often lack robust theoretical interpretability. Conversely, traditional iterative optimization algorithms are built upon sound mathematical derivations. To combine the advantages of both approaches, we propose a spectral transformer network, termed STTODNet, which integrates deep tensor decomposition and omni-dimensional dynamic convolution (ODConv). Specifically, we incorporate a deep Tucker decomposition module within the self-attention mechanism to effectively extract low-rank prior features inherent in the hyperspectral image. Moreover, we replace the conventional linear projection layer with ODConv to substantially improve feature extraction capabilities. A three-scale U-Net network structure is designed as the approximate operator for solving the prior within our deep unfolding network architecture. Extensive experimental results demonstrate that STTODNet achieves superior results in terms of reconstruction quality, interpretability, and computational efficiency when compared to state-of-the-art methods.https://ieeexplore.ieee.org/document/11022735/Compressive imagingdeep Tucker decomposition (DTD)omni-dimensional dynamic convolution (ODConv)transformer |
| spellingShingle | Le Sun Xihan Ma Xinyu Wang Qiao Chen Zebin Wu HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Compressive imaging deep Tucker decomposition (DTD) omni-dimensional dynamic convolution (ODConv) transformer |
| title | HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution |
| title_full | HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution |
| title_fullStr | HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution |
| title_full_unstemmed | HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution |
| title_short | HSI Reconstruction: A Spectral Transformer With Tensor Decomposition and Dynamic Convolution |
| title_sort | hsi reconstruction a spectral transformer with tensor decomposition and dynamic convolution |
| topic | Compressive imaging deep Tucker decomposition (DTD) omni-dimensional dynamic convolution (ODConv) transformer |
| url | https://ieeexplore.ieee.org/document/11022735/ |
| work_keys_str_mv | AT lesun hsireconstructionaspectraltransformerwithtensordecompositionanddynamicconvolution AT xihanma hsireconstructionaspectraltransformerwithtensordecompositionanddynamicconvolution AT xinyuwang hsireconstructionaspectraltransformerwithtensordecompositionanddynamicconvolution AT qiaochen hsireconstructionaspectraltransformerwithtensordecompositionanddynamicconvolution AT zebinwu hsireconstructionaspectraltransformerwithtensordecompositionanddynamicconvolution |