Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework
Fine-tuning pretrained models for remote sensing tasks often demands substantial computational resources. To reduce memory requirements and training costs, this article proposes a low-memory fine-tuning framework, called efficient side-tuning (EST), for remote sensing downstream tasks. EST attaches...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10974700/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850138186389389312 |
|---|---|
| author | Haichen Yu Wenxin Yin Hanbo Bi Chongyang Li Yingchao Feng Wenhui Diao Xian Sun |
| author_facet | Haichen Yu Wenxin Yin Hanbo Bi Chongyang Li Yingchao Feng Wenhui Diao Xian Sun |
| author_sort | Haichen Yu |
| collection | DOAJ |
| description | Fine-tuning pretrained models for remote sensing tasks often demands substantial computational resources. To reduce memory requirements and training costs, this article proposes a low-memory fine-tuning framework, called efficient side-tuning (EST), for remote sensing downstream tasks. EST attaches a parallel network to the backbone of the model, and only fine-tunes the parameters of the parallel network during the training phase. The proposed EST Block is the main component of the parallel network, which uses the multichannel adapter fusion module, gate layer and depthwise convolution to achieve feature selection and enhancement effects. In the evaluation, on six remote sensing datasets including object detection and semantic segmentation, EST achieved SOTA performance results using only less than 40<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> of the memory expenditure of full fine-tuning, which is better than all current parameter efficient fine-tuning methods. In addition, experiments on backbones of various sizes and classes show that the generalizability of EST is also reliable. EST thus offers a highly efficient and effective approach for efficient transfer learning in remote sensing, unlocking new possibilities for advanced remote sensing applications. |
| format | Article |
| id | doaj-art-e2995fe9b8e24264870c148dd5a56f65 |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-e2995fe9b8e24264870c148dd5a56f652025-08-20T02:30:38ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118119081192510.1109/JSTARS.2025.356364110974700Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning FrameworkHaichen Yu0https://orcid.org/0009-0001-8123-417XWenxin Yin1https://orcid.org/0000-0003-0157-7947Hanbo Bi2https://orcid.org/0009-0001-4209-5461Chongyang Li3https://orcid.org/0009-0003-8234-4420Yingchao Feng4https://orcid.org/0000-0003-4017-8885Wenhui Diao5https://orcid.org/0000-0002-3931-3974Xian Sun6https://orcid.org/0000-0002-0038-9816Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaFine-tuning pretrained models for remote sensing tasks often demands substantial computational resources. To reduce memory requirements and training costs, this article proposes a low-memory fine-tuning framework, called efficient side-tuning (EST), for remote sensing downstream tasks. EST attaches a parallel network to the backbone of the model, and only fine-tunes the parameters of the parallel network during the training phase. The proposed EST Block is the main component of the parallel network, which uses the multichannel adapter fusion module, gate layer and depthwise convolution to achieve feature selection and enhancement effects. In the evaluation, on six remote sensing datasets including object detection and semantic segmentation, EST achieved SOTA performance results using only less than 40<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> of the memory expenditure of full fine-tuning, which is better than all current parameter efficient fine-tuning methods. In addition, experiments on backbones of various sizes and classes show that the generalizability of EST is also reliable. EST thus offers a highly efficient and effective approach for efficient transfer learning in remote sensing, unlocking new possibilities for advanced remote sensing applications.https://ieeexplore.ieee.org/document/10974700/Fine-tuningmemory efficienctobject detectionremote sensingsemantic segmentation |
| spellingShingle | Haichen Yu Wenxin Yin Hanbo Bi Chongyang Li Yingchao Feng Wenhui Diao Xian Sun Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Fine-tuning memory efficienct object detection remote sensing semantic segmentation |
| title | Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework |
| title_full | Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework |
| title_fullStr | Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework |
| title_full_unstemmed | Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework |
| title_short | Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework |
| title_sort | efficient side tuning for remote sensing a low memory fine tuning framework |
| topic | Fine-tuning memory efficienct object detection remote sensing semantic segmentation |
| url | https://ieeexplore.ieee.org/document/10974700/ |
| work_keys_str_mv | AT haichenyu efficientsidetuningforremotesensingalowmemoryfinetuningframework AT wenxinyin efficientsidetuningforremotesensingalowmemoryfinetuningframework AT hanbobi efficientsidetuningforremotesensingalowmemoryfinetuningframework AT chongyangli efficientsidetuningforremotesensingalowmemoryfinetuningframework AT yingchaofeng efficientsidetuningforremotesensingalowmemoryfinetuningframework AT wenhuidiao efficientsidetuningforremotesensingalowmemoryfinetuningframework AT xiansun efficientsidetuningforremotesensingalowmemoryfinetuningframework |