Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification
The combination of convolutional neural networks and vision transformers has garnered considerable attention in hyperspectral image (HSI) classification due to their abilities to enhance the classification accuracy by concurrently extracting local and global features. However, these accuracy improve...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10850760/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849240425486876672 |
|---|---|
| author | Qinggang Wu Mengkun He Qiqiang Chen Le Sun Chao Ma |
| author_facet | Qinggang Wu Mengkun He Qiqiang Chen Le Sun Chao Ma |
| author_sort | Qinggang Wu |
| collection | DOAJ |
| description | The combination of convolutional neural networks and vision transformers has garnered considerable attention in hyperspectral image (HSI) classification due to their abilities to enhance the classification accuracy by concurrently extracting local and global features. However, these accuracy improvements come at the cost of significant demands on storage resources, computational overhead, and extensive training samples. To address these challenges, this article proposes a multiscale spatial–spectral shuffling convolution integrated with a 3-D lightweight transformer (MSC-3DLT) for HSI classification. This network directly captures 3-D structural features throughout the entire feature extraction process, thereby enhancing HSI classification performance even at small sampling rates within a lightweight framework. Specifically, we first design a multiscale spatial–spectral shuffling convolution to comprehensively refine spatial–spectral feature granularities and enhance feature interactions by shuffling multiscale features across different groups. Second, to maximize the exploitation of limited training samples, we rethink transformers from the 3-D structural perspective of HSI data and propose a novel 3-D lightweight transformer (3DLT). Different from the slicing operation employed in classical transformers, the 3DLT directly extracts the inherent 3-D structural features from the HSI and mitigates quadratic complexity through a lightweight spatial–spectral pooling cross-attention mechanism. Finally, a novel training strategy is designed to adaptively adjust the learning rate based on multimetric feedback during the model training process, significantly accelerating the model fitting speed. Extensive experiments demonstrate that the proposed MSC-3DLT method remains highly competitive compared with state-of-the-art methods in terms of classification accuracy, model parameters, and floating point and operations under small sampling rates. |
| format | Article |
| id | doaj-art-4e752bb4463e4f8d86ce13ac0415aee1 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-4e752bb4463e4f8d86ce13ac0415aee12025-08-20T04:00:34ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01185378539410.1109/JSTARS.2025.353321110850760Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image ClassificationQinggang Wu0https://orcid.org/0009-0008-6789-8472Mengkun He1Qiqiang Chen2https://orcid.org/0009-0005-5965-7853Le Sun3https://orcid.org/0000-0001-6465-8678Chao Ma4College of Computer Science and Technology, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Computer Science and Technology, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Computer Science and Technology, Zhengzhou University of Light Industry, Zhengzhou, ChinaSchool of Computer Science and the Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing, ChinaCollege of Computer Science and Technology, Zhengzhou University of Light Industry, Zhengzhou, ChinaThe combination of convolutional neural networks and vision transformers has garnered considerable attention in hyperspectral image (HSI) classification due to their abilities to enhance the classification accuracy by concurrently extracting local and global features. However, these accuracy improvements come at the cost of significant demands on storage resources, computational overhead, and extensive training samples. To address these challenges, this article proposes a multiscale spatial–spectral shuffling convolution integrated with a 3-D lightweight transformer (MSC-3DLT) for HSI classification. This network directly captures 3-D structural features throughout the entire feature extraction process, thereby enhancing HSI classification performance even at small sampling rates within a lightweight framework. Specifically, we first design a multiscale spatial–spectral shuffling convolution to comprehensively refine spatial–spectral feature granularities and enhance feature interactions by shuffling multiscale features across different groups. Second, to maximize the exploitation of limited training samples, we rethink transformers from the 3-D structural perspective of HSI data and propose a novel 3-D lightweight transformer (3DLT). Different from the slicing operation employed in classical transformers, the 3DLT directly extracts the inherent 3-D structural features from the HSI and mitigates quadratic complexity through a lightweight spatial–spectral pooling cross-attention mechanism. Finally, a novel training strategy is designed to adaptively adjust the learning rate based on multimetric feedback during the model training process, significantly accelerating the model fitting speed. Extensive experiments demonstrate that the proposed MSC-3DLT method remains highly competitive compared with state-of-the-art methods in terms of classification accuracy, model parameters, and floating point and operations under small sampling rates.https://ieeexplore.ieee.org/document/10850760/Convolutional neural networks (CNNs)hyperspectral image (HSI) classificationmultimetric adaptive learning rate (MALR)3-D lightweight transformer (3DLT) |
| spellingShingle | Qinggang Wu Mengkun He Qiqiang Chen Le Sun Chao Ma Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Convolutional neural networks (CNNs) hyperspectral image (HSI) classification multimetric adaptive learning rate (MALR) 3-D lightweight transformer (3DLT) |
| title | Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification |
| title_full | Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification |
| title_fullStr | Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification |
| title_full_unstemmed | Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification |
| title_short | Integrating Multiscale Spatial–Spectral Shuffling Convolution With 3-D Lightweight Transformer for Hyperspectral Image Classification |
| title_sort | integrating multiscale spatial x2013 spectral shuffling convolution with 3 d lightweight transformer for hyperspectral image classification |
| topic | Convolutional neural networks (CNNs) hyperspectral image (HSI) classification multimetric adaptive learning rate (MALR) 3-D lightweight transformer (3DLT) |
| url | https://ieeexplore.ieee.org/document/10850760/ |
| work_keys_str_mv | AT qinggangwu integratingmultiscalespatialx2013spectralshufflingconvolutionwith3dlightweighttransformerforhyperspectralimageclassification AT mengkunhe integratingmultiscalespatialx2013spectralshufflingconvolutionwith3dlightweighttransformerforhyperspectralimageclassification AT qiqiangchen integratingmultiscalespatialx2013spectralshufflingconvolutionwith3dlightweighttransformerforhyperspectralimageclassification AT lesun integratingmultiscalespatialx2013spectralshufflingconvolutionwith3dlightweighttransformerforhyperspectralimageclassification AT chaoma integratingmultiscalespatialx2013spectralshufflingconvolutionwith3dlightweighttransformerforhyperspectralimageclassification |