DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion
Remote sensing image spatiotemporal fusion (STF) aims to generate composite images with high-temporal and spatial resolutions by combining remote sensing images captured at different times and with different spatial resolutions (DTDS). Among the existing fusion algorithms, deep learning-based fusion...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10707182/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850150717758636032 |
|---|---|
| author | Yan Zhang Rongbo Fan PeiPei Duan Jinfang Dong Zhiyong Lei |
| author_facet | Yan Zhang Rongbo Fan PeiPei Duan Jinfang Dong Zhiyong Lei |
| author_sort | Yan Zhang |
| collection | DOAJ |
| description | Remote sensing image spatiotemporal fusion (STF) aims to generate composite images with high-temporal and spatial resolutions by combining remote sensing images captured at different times and with different spatial resolutions (DTDS). Among the existing fusion algorithms, deep learning-based fusion models have demonstrated outstanding performance. These models treat STF as an image super-resolution problem based on multiple reference images. However, compared to traditional image super-resolution tasks, remote sensing image STF involves merging a larger amount of multitemporal data with greater resolution difference. To enhance the robust matching performance of spatiotemporal transformations between multiple sets of remote sensing images captured at DTDS and to generate super-resolution composite images, we propose a feature fusion network called the multiscale deformable convolution distillation generative adversarial network (DCDGAN-STF). Specifically, to address the differences in multitemporal data, we introduce a pyramid cascading deformable encoder to identify disparities in multitemporal images. In addition, to address the differences in spatial resolution, we propose a teacher–student correlation distillation method. This method uses the texture details' disparities between high-resolution multitemporal images to guide the extraction of disparities in blurred low-resolution multitemporal images. We comprehensively compared the proposed DCDGAN-STF with some state-of-the-art algorithms on two landsat and moderate-resolution imaging spectroradiometer datasets. Ablation experiments were also conducted to test the effectiveness of different submodules within DCDGAN-STF. The experimental results and ablation analysis demonstrate that our algorithm achieves superior performance compared to other algorithms. |
| format | Article |
| id | doaj-art-045a5400aa3a49f4b75de61a79ccd30b |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-045a5400aa3a49f4b75de61a79ccd30b2025-08-20T02:26:28ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352024-01-0117194361945010.1109/JSTARS.2024.347615310707182DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal FusionYan Zhang0https://orcid.org/0000-0002-4636-9386Rongbo Fan1https://orcid.org/0000-0003-3284-9685PeiPei Duan2Jinfang Dong3Zhiyong Lei4https://orcid.org/0009-0008-9805-5136School of Mechatronic Engineering, Xi'an Technological University, Xi'an, ChinaSchool of Automation, Northwestern Polytechnical University, Xi'an, ChinaSchool of Computer Science, Xi'an Shiyou University, Xi'an, ChinaShaanxi Meteorological Service Center of Agricultural Remote Sensing and Economic Crops, Xi'an, ChinaSchool of Electronic and Information Engineering, Xi'an Technological University, Xi'an, ChinaRemote sensing image spatiotemporal fusion (STF) aims to generate composite images with high-temporal and spatial resolutions by combining remote sensing images captured at different times and with different spatial resolutions (DTDS). Among the existing fusion algorithms, deep learning-based fusion models have demonstrated outstanding performance. These models treat STF as an image super-resolution problem based on multiple reference images. However, compared to traditional image super-resolution tasks, remote sensing image STF involves merging a larger amount of multitemporal data with greater resolution difference. To enhance the robust matching performance of spatiotemporal transformations between multiple sets of remote sensing images captured at DTDS and to generate super-resolution composite images, we propose a feature fusion network called the multiscale deformable convolution distillation generative adversarial network (DCDGAN-STF). Specifically, to address the differences in multitemporal data, we introduce a pyramid cascading deformable encoder to identify disparities in multitemporal images. In addition, to address the differences in spatial resolution, we propose a teacher–student correlation distillation method. This method uses the texture details' disparities between high-resolution multitemporal images to guide the extraction of disparities in blurred low-resolution multitemporal images. We comprehensively compared the proposed DCDGAN-STF with some state-of-the-art algorithms on two landsat and moderate-resolution imaging spectroradiometer datasets. Ablation experiments were also conducted to test the effectiveness of different submodules within DCDGAN-STF. The experimental results and ablation analysis demonstrate that our algorithm achieves superior performance compared to other algorithms.https://ieeexplore.ieee.org/document/10707182/Deformable convolutiongenerative adversarial network (GAN)remote sensing image spatiotemporal fusion (STF)teacher–student correlation distillation |
| spellingShingle | Yan Zhang Rongbo Fan PeiPei Duan Jinfang Dong Zhiyong Lei DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Deformable convolution generative adversarial network (GAN) remote sensing image spatiotemporal fusion (STF) teacher–student correlation distillation |
| title | DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion |
| title_full | DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion |
| title_fullStr | DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion |
| title_full_unstemmed | DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion |
| title_short | DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion |
| title_sort | dcdgan stf a multiscale deformable convolution distillation gan for remote sensing image spatiotemporal fusion |
| topic | Deformable convolution generative adversarial network (GAN) remote sensing image spatiotemporal fusion (STF) teacher–student correlation distillation |
| url | https://ieeexplore.ieee.org/document/10707182/ |
| work_keys_str_mv | AT yanzhang dcdganstfamultiscaledeformableconvolutiondistillationganforremotesensingimagespatiotemporalfusion AT rongbofan dcdganstfamultiscaledeformableconvolutiondistillationganforremotesensingimagespatiotemporalfusion AT peipeiduan dcdganstfamultiscaledeformableconvolutiondistillationganforremotesensingimagespatiotemporalfusion AT jinfangdong dcdganstfamultiscaledeformableconvolutiondistillationganforremotesensingimagespatiotemporalfusion AT zhiyonglei dcdganstfamultiscaledeformableconvolutiondistillationganforremotesensingimagespatiotemporalfusion |