DMPNet: dual-path and multi-scale pansharpening network
IntroductionPansharpening is an important remote sensing task that aims to produce high-resolution multispectral (MS) images by combining low-resolution MS images with high-resolution panchromatic (PAN) images. Although deep learning-based pansharpening has shown impressive results, the majority of...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2024.1455963/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841525833819226112 |
|---|---|
| author | Gurpreet Kaur Manisha Malhotra Dilbag Singh Dilbag Singh Sunita Singhal |
| author_facet | Gurpreet Kaur Manisha Malhotra Dilbag Singh Dilbag Singh Sunita Singhal |
| author_sort | Gurpreet Kaur |
| collection | DOAJ |
| description | IntroductionPansharpening is an important remote sensing task that aims to produce high-resolution multispectral (MS) images by combining low-resolution MS images with high-resolution panchromatic (PAN) images. Although deep learning-based pansharpening has shown impressive results, the majority of these models frequently struggle to balance spatial and spectral information, resulting in artifacts and a loss of detail in pansharpened images. Furthermore, these models may fail to properly integrate spatial and spectral information, leading to poor performance in complex scenarios. Additionally, these models face challenges such as gradient vanishing and overfitting.MethodsThis paper proposes a dual-path and multi-scale pansharpening network (DMPNet). It consists of three modules: the feature extraction module (FEM), the multi-scale adaptive attention fusion module (MSAAF), and the image reconstruction module (IRM). The FEM is designed with two paths, namely the primary and secondary paths. The primary path captures global spatial and spectral information using dilated convolutions, while the secondary path focuses on fine-grained details using shallow convolutions and attention-guided feature extraction. The MSAAF module adaptively combines spatial and spectral data across different scales, employing a self-calibrated attention (SCA) mechanism for dynamic weighting of local and global contexts and a spectral alignment network (SAN) to ensure spectral consistency. Finally, to achieve optimal spatial and spectral reconstruction, the IRM decomposes the fused features into low- and high-frequency components using discrete wavelet transform (DWT).ResultsThe proposed DMPNet outperforms competitive models in terms of ERGAS, SCC (WR), SCC (NR), PSNR, Q, QNR, and JQM by approximately 1.24%, 1.18%, 1.37%, 1.42%, 1.26%, 1.31%, and 1.23%, respectively.DiscussionExtensive experimental results and evaluations reveal that the DMPNet is more efficient and robust than competing pansharpening models. |
| format | Article |
| id | doaj-art-25de9967033a4ee5a302c2150bf1e6f7 |
| institution | Kabale University |
| issn | 2624-9898 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj-art-25de9967033a4ee5a302c2150bf1e6f72025-01-17T06:51:12ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982025-01-01610.3389/fcomp.2024.14559631455963DMPNet: dual-path and multi-scale pansharpening networkGurpreet Kaur0Manisha Malhotra1Dilbag Singh2Dilbag Singh3Sunita Singhal4University Institute of Computing, Chandigarh University, Gharuan, IndiaUniversity Institute of Computing, Chandigarh University, Gharuan, IndiaCentre of Research Impact and Outcome, Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, IndiaResearch and Development Cell, Lovely Professional University, Phagwara, IndiaDepartment of Computer Science Engineering, Manipal University Jaipur, Jaipur, IndiaIntroductionPansharpening is an important remote sensing task that aims to produce high-resolution multispectral (MS) images by combining low-resolution MS images with high-resolution panchromatic (PAN) images. Although deep learning-based pansharpening has shown impressive results, the majority of these models frequently struggle to balance spatial and spectral information, resulting in artifacts and a loss of detail in pansharpened images. Furthermore, these models may fail to properly integrate spatial and spectral information, leading to poor performance in complex scenarios. Additionally, these models face challenges such as gradient vanishing and overfitting.MethodsThis paper proposes a dual-path and multi-scale pansharpening network (DMPNet). It consists of three modules: the feature extraction module (FEM), the multi-scale adaptive attention fusion module (MSAAF), and the image reconstruction module (IRM). The FEM is designed with two paths, namely the primary and secondary paths. The primary path captures global spatial and spectral information using dilated convolutions, while the secondary path focuses on fine-grained details using shallow convolutions and attention-guided feature extraction. The MSAAF module adaptively combines spatial and spectral data across different scales, employing a self-calibrated attention (SCA) mechanism for dynamic weighting of local and global contexts and a spectral alignment network (SAN) to ensure spectral consistency. Finally, to achieve optimal spatial and spectral reconstruction, the IRM decomposes the fused features into low- and high-frequency components using discrete wavelet transform (DWT).ResultsThe proposed DMPNet outperforms competitive models in terms of ERGAS, SCC (WR), SCC (NR), PSNR, Q, QNR, and JQM by approximately 1.24%, 1.18%, 1.37%, 1.42%, 1.26%, 1.31%, and 1.23%, respectively.DiscussionExtensive experimental results and evaluations reveal that the DMPNet is more efficient and robust than competing pansharpening models.https://www.frontiersin.org/articles/10.3389/fcomp.2024.1455963/fullpansharpeningremote sensingdeep learningimage reconstructionspatial and spectral fidelity |
| spellingShingle | Gurpreet Kaur Manisha Malhotra Dilbag Singh Dilbag Singh Sunita Singhal DMPNet: dual-path and multi-scale pansharpening network Frontiers in Computer Science pansharpening remote sensing deep learning image reconstruction spatial and spectral fidelity |
| title | DMPNet: dual-path and multi-scale pansharpening network |
| title_full | DMPNet: dual-path and multi-scale pansharpening network |
| title_fullStr | DMPNet: dual-path and multi-scale pansharpening network |
| title_full_unstemmed | DMPNet: dual-path and multi-scale pansharpening network |
| title_short | DMPNet: dual-path and multi-scale pansharpening network |
| title_sort | dmpnet dual path and multi scale pansharpening network |
| topic | pansharpening remote sensing deep learning image reconstruction spatial and spectral fidelity |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2024.1455963/full |
| work_keys_str_mv | AT gurpreetkaur dmpnetdualpathandmultiscalepansharpeningnetwork AT manishamalhotra dmpnetdualpathandmultiscalepansharpeningnetwork AT dilbagsingh dmpnetdualpathandmultiscalepansharpeningnetwork AT dilbagsingh dmpnetdualpathandmultiscalepansharpeningnetwork AT sunitasinghal dmpnetdualpathandmultiscalepansharpeningnetwork |