Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration
Infrared and visible image registration ensures consistency in spatial positions across different modalities. Cross-modal images contain different scales objects and cluttered backgrounds. However, most existing image registration methods adopt the same alignment strategy for different objects, whic...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10833818/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832583224955502592 |
|---|---|
| author | Chunna Tian Liuwei Xu Xiangyang Li Heng Zhou Xiqun Song |
| author_facet | Chunna Tian Liuwei Xu Xiangyang Li Heng Zhou Xiqun Song |
| author_sort | Chunna Tian |
| collection | DOAJ |
| description | Infrared and visible image registration ensures consistency in spatial positions across different modalities. Cross-modal images contain different scales objects and cluttered backgrounds. However, most existing image registration methods adopt the same alignment strategy for different objects, which leads to insufficient multiscale feature representation and inaccurate registration of foreground objects. To address these issues, we propose a semantic-injected bidirectional multiscale flow estimation (SI-BMFE) network for infrared and visible image registration. SI-BMFE leverages feature complementarity across different scales and employs a pretrained segmentation network to extract the spatial positions of foreground objects to improve registration accuracy. Specifically, we first design a bidirectional multiscale feature enhancement (BMFE) module to integrate feature complementarity across different scales, effectively extracts both global structures and local details. BMFE pushes the network to roughly align infrared and visible images. Then, the semantic-injected flow estimation (SFE) module is introduced to estimate multilevel deformation fields for fine-grained registration of different objects. SFE utilizes a pretrained segmentation network to obtain spatial location information of foreground objects. Object location cues help the network distinguish and focus on different foreground objects from the background. SFE exploits semantic knowledge to promote fine alignment of different foreground objects and improve the accuracy of cross-modal image registration. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art registration networks on both the MSRS and RoadScene infrared and visible image registration datasets. |
| format | Article |
| id | doaj-art-57bed21aeccb498e906e416f8e6ff552 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-57bed21aeccb498e906e416f8e6ff5522025-01-29T00:00:04ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01183686369510.1109/JSTARS.2025.352717510833818Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image RegistrationChunna Tian0https://orcid.org/0000-0002-3217-0368Liuwei Xu1Xiangyang Li2https://orcid.org/0009-0003-8540-6009Heng Zhou3https://orcid.org/0000-0003-2770-2785Xiqun Song4https://orcid.org/0009-0005-1929-914XSchool of Electronic Engineering, Xidian University, Xi'an, ChinaSchool of Electronic Engineering, Xidian University, Xi'an, ChinaSchool of Electronic Engineering, Xidian University, Xi'an, ChinaSchool of Artificial Intelligence and Computer Science, and the Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, Jiangnan University, Wuxi, ChinaSchool of Communication and Information Engineering, Xi'an University of Posts and Telecommunications, Xi'an, ChinaInfrared and visible image registration ensures consistency in spatial positions across different modalities. Cross-modal images contain different scales objects and cluttered backgrounds. However, most existing image registration methods adopt the same alignment strategy for different objects, which leads to insufficient multiscale feature representation and inaccurate registration of foreground objects. To address these issues, we propose a semantic-injected bidirectional multiscale flow estimation (SI-BMFE) network for infrared and visible image registration. SI-BMFE leverages feature complementarity across different scales and employs a pretrained segmentation network to extract the spatial positions of foreground objects to improve registration accuracy. Specifically, we first design a bidirectional multiscale feature enhancement (BMFE) module to integrate feature complementarity across different scales, effectively extracts both global structures and local details. BMFE pushes the network to roughly align infrared and visible images. Then, the semantic-injected flow estimation (SFE) module is introduced to estimate multilevel deformation fields for fine-grained registration of different objects. SFE utilizes a pretrained segmentation network to obtain spatial location information of foreground objects. Object location cues help the network distinguish and focus on different foreground objects from the background. SFE exploits semantic knowledge to promote fine alignment of different foreground objects and improve the accuracy of cross-modal image registration. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art registration networks on both the MSRS and RoadScene infrared and visible image registration datasets.https://ieeexplore.ieee.org/document/10833818/Flow estimationimage registrationinfrared and visible imagesemantic-injected |
| spellingShingle | Chunna Tian Liuwei Xu Xiangyang Li Heng Zhou Xiqun Song Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Flow estimation image registration infrared and visible image semantic-injected |
| title | Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration |
| title_full | Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration |
| title_fullStr | Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration |
| title_full_unstemmed | Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration |
| title_short | Semantic-Injected Bidirectional Multiscale Flow Estimation Network for Infrared and Visible Image Registration |
| title_sort | semantic injected bidirectional multiscale flow estimation network for infrared and visible image registration |
| topic | Flow estimation image registration infrared and visible image semantic-injected |
| url | https://ieeexplore.ieee.org/document/10833818/ |
| work_keys_str_mv | AT chunnatian semanticinjectedbidirectionalmultiscaleflowestimationnetworkforinfraredandvisibleimageregistration AT liuweixu semanticinjectedbidirectionalmultiscaleflowestimationnetworkforinfraredandvisibleimageregistration AT xiangyangli semanticinjectedbidirectionalmultiscaleflowestimationnetworkforinfraredandvisibleimageregistration AT hengzhou semanticinjectedbidirectionalmultiscaleflowestimationnetworkforinfraredandvisibleimageregistration AT xiqunsong semanticinjectedbidirectionalmultiscaleflowestimationnetworkforinfraredandvisibleimageregistration |