PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features
Geometric distortions and significant nonlinear radiometric differences in multimodal remote sensing images (MRSIs) introduce substantial noise in feature extraction. Single-branch convolutional neural networks fail to capture global image features and integrate local and global information effectiv...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10999084/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850232654641758208 |
|---|---|
| author | Haiqing He Shixun Yu Yongjun Zhang Yufeng Zhu Ting Chen Fuyang Zhou |
| author_facet | Haiqing He Shixun Yu Yongjun Zhang Yufeng Zhu Ting Chen Fuyang Zhou |
| author_sort | Haiqing He |
| collection | DOAJ |
| description | Geometric distortions and significant nonlinear radiometric differences in multimodal remote sensing images (MRSIs) introduce substantial noise in feature extraction. Single-branch convolutional neural networks fail to capture global image features and integrate local and global information effectively, yielding deep descriptors with low discriminability and limited robustness. Moreover, the lack of comprehensive training data further limits the network's performance, which poses a formidable challenge to existing matching methods in securing adequate and evenly distributed corresponding points. This article proposes a novel method called phase-integrated aggregated deep feature matching (PI-ADFM), designed to address these challenges. Initially, a phase structure feature detector is introduced, which amalgamates the structural attributes and phase information of images to distill keypoints that are highly repeatable and exhibit minimal redundancy. Subsequently, an attention-based multilevel feature interaction and aggregation module is crafted to encapsulate a comprehensive representation of both local and global features of keypoints. This is followed by the integration of a dense feature fusion module designed to sift through and amalgamate key features, thereby capturing highly discriminative deep semantic features that serve as descriptors for similarity measures. Finally, a multilevel outlier removal strategy is proposed to effectively reduce mismatches. Experimental results substantiate that, in juxtaposition with state-of-the-art methods, the PI-ADFM method has significantly augmented the count of matches for optical-infrared and optical synthetic aperture radar images by a factor of at least 1.7 and 3.7, respectively, while concurrently enhancing the accuracy by a minimum of 10% and 6%, respectively. These enhancements markedly bolster the robustness and reliability of MRSI matching endeavors. |
| format | Article |
| id | doaj-art-80fa3a545896483c81f2f0876b914d9c |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-80fa3a545896483c81f2f0876b914d9c2025-08-20T02:03:07ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118131921321110.1109/JSTARS.2025.356917410999084PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep FeaturesHaiqing He0https://orcid.org/0000-0001-9361-0219Shixun Yu1https://orcid.org/0009-0001-5835-0949Yongjun Zhang2https://orcid.org/0000-0001-9845-4251Yufeng Zhu3Ting Chen4Fuyang Zhou5https://orcid.org/0009-0005-5394-0762National Key Laboratory of Uranium Resources Exploration-Mining and Nuclear Remote Sensing, East China University of Technology, Nanchang, ChinaNational Key Laboratory of Uranium Resources Exploration-Mining and Nuclear Remote Sensing, East China University of Technology, Nanchang, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaNational Key Laboratory of Uranium Resources Exploration-Mining and Nuclear Remote Sensing, East China University of Technology, Nanchang, ChinaSchool of Water Resources and Environmental Engineering, East China University of Technology, Nanchang, ChinaNational Key Laboratory of Uranium Resources Exploration-Mining and Nuclear Remote Sensing, East China University of Technology, Nanchang, ChinaGeometric distortions and significant nonlinear radiometric differences in multimodal remote sensing images (MRSIs) introduce substantial noise in feature extraction. Single-branch convolutional neural networks fail to capture global image features and integrate local and global information effectively, yielding deep descriptors with low discriminability and limited robustness. Moreover, the lack of comprehensive training data further limits the network's performance, which poses a formidable challenge to existing matching methods in securing adequate and evenly distributed corresponding points. This article proposes a novel method called phase-integrated aggregated deep feature matching (PI-ADFM), designed to address these challenges. Initially, a phase structure feature detector is introduced, which amalgamates the structural attributes and phase information of images to distill keypoints that are highly repeatable and exhibit minimal redundancy. Subsequently, an attention-based multilevel feature interaction and aggregation module is crafted to encapsulate a comprehensive representation of both local and global features of keypoints. This is followed by the integration of a dense feature fusion module designed to sift through and amalgamate key features, thereby capturing highly discriminative deep semantic features that serve as descriptors for similarity measures. Finally, a multilevel outlier removal strategy is proposed to effectively reduce mismatches. Experimental results substantiate that, in juxtaposition with state-of-the-art methods, the PI-ADFM method has significantly augmented the count of matches for optical-infrared and optical synthetic aperture radar images by a factor of at least 1.7 and 3.7, respectively, while concurrently enhancing the accuracy by a minimum of 10% and 6%, respectively. These enhancements markedly bolster the robustness and reliability of MRSI matching endeavors.https://ieeexplore.ieee.org/document/10999084/Deep learningfeature interaction and aggregationfeature matchingmultimodal imagesphase structure |
| spellingShingle | Haiqing He Shixun Yu Yongjun Zhang Yufeng Zhu Ting Chen Fuyang Zhou PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Deep learning feature interaction and aggregation feature matching multimodal images phase structure |
| title | PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features |
| title_full | PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features |
| title_fullStr | PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features |
| title_full_unstemmed | PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features |
| title_short | PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features |
| title_sort | pi adfm enhancing multimodal remote sensing image matching through phase integrated aggregated deep features |
| topic | Deep learning feature interaction and aggregation feature matching multimodal images phase structure |
| url | https://ieeexplore.ieee.org/document/10999084/ |
| work_keys_str_mv | AT haiqinghe piadfmenhancingmultimodalremotesensingimagematchingthroughphaseintegratedaggregateddeepfeatures AT shixunyu piadfmenhancingmultimodalremotesensingimagematchingthroughphaseintegratedaggregateddeepfeatures AT yongjunzhang piadfmenhancingmultimodalremotesensingimagematchingthroughphaseintegratedaggregateddeepfeatures AT yufengzhu piadfmenhancingmultimodalremotesensingimagematchingthroughphaseintegratedaggregateddeepfeatures AT tingchen piadfmenhancingmultimodalremotesensingimagematchingthroughphaseintegratedaggregateddeepfeatures AT fuyangzhou piadfmenhancingmultimodalremotesensingimagematchingthroughphaseintegratedaggregateddeepfeatures |