Curriculum Preview and Review Based on Knowledge Distillation for Hyperspectral Image Classification
Applying knowledge distillation to hyperspectral image (HSI) classification has become a significant research area, aiming to develop a student network with low computational complexity and high classification performance. However, the traditional distillation process involves unordered knowledge tr...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11061777/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Applying knowledge distillation to hyperspectral image (HSI) classification has become a significant research area, aiming to develop a student network with low computational complexity and high classification performance. However, the traditional distillation process involves unordered knowledge transfer, which is suboptimal for the learning of the student network. Therefore, this article integrates human education principles with the rich spatial-spectral information of HSI and proposes curriculum preview and curriculum review based on knowledge distillation. The aim is to ensure orderly knowledge transfer during the distillation process, enabling the student network to progressively learn HSI features and thereby further enhance the distillation effectiveness. In the curriculum preview phase, this process is achieved by applying the momentum contrast (MOCO) method to the feature extraction layers of the student network. In the curriculum review phase, this article proposes a dynamic, learnable, and instance-level review coefficient to assist the student network in completing the review task. In the process of human education, the review process for students follows a progression from easy to difficult. Therefore, in our method, review coefficients are gradually increased in an adversarial manner to ensure that the review progression of student network transitions from easy to difficult. At the same time, by using a queue to store and combine the logits from the previous epoch with those from the current epoch, the curriculum learning process is ensured to follow a progression from easy to difficult. The aforementioned preview and review methods can be easily integrated into other logits-based knowledge distillation frameworks to enhance student network performance. Experiments conducted on four publicly available HSI datasets demonstrate the necessity and effectiveness of curriculum preview and curriculum review, significantly improving the classification performance of the student network. |
|---|---|
| ISSN: | 1939-1404 2151-1535 |