Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks
In order to solve the problem that existing visible and infrared image fusion methods rely only on the original local or global information representation, which has the problem of edge blurring and non-protrusion of salient targets, this paper proposes a layered fusion method based on channel atten...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/21/6916 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850197012823146496 |
|---|---|
| author | Jie Wu Shuai Yang Xiaoming Wang Yu Pei Shuai Wang Congcong Song |
| author_facet | Jie Wu Shuai Yang Xiaoming Wang Yu Pei Shuai Wang Congcong Song |
| author_sort | Jie Wu |
| collection | DOAJ |
| description | In order to solve the problem that existing visible and infrared image fusion methods rely only on the original local or global information representation, which has the problem of edge blurring and non-protrusion of salient targets, this paper proposes a layered fusion method based on channel attention mechanism and improved Generative Adversarial Network (HFCA_GAN). Firstly, the infrared image and visible image are decomposed into a base layer and fine layer, respectively, by a guiding filter. Secondly, the visible light base layer is fused with the infrared image base layer by histogram mapping enhancement to improve the contour effect. Thirdly, the improved GAN algorithm is used to fuse the infrared and visible image refinement layer, and the depth transferable module and guided fusion network are added to enrich the detailed information of the fused image. Finally, the multilayer convolutional fusion network with channel attention mechanism is used to correlate the local information of the layered fusion image, and the final fusion image containing contour gradient information and useful details is obtained. TNO and RoadSence datasets are selected for training and testing. The results show that the proposed algorithm retains the global structure features of multilayer images and has obvious advantages in fusion performance, model generalization and computational efficiency. |
| format | Article |
| id | doaj-art-8aadadcc0cc44902a00f0afafb353f7f |
| institution | OA Journals |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-8aadadcc0cc44902a00f0afafb353f7f2025-08-20T02:13:19ZengMDPI AGSensors1424-82202024-10-012421691610.3390/s24216916Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial NetworksJie Wu0Shuai Yang1Xiaoming Wang2Yu Pei3Shuai Wang4Congcong Song5Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaSuzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaIn order to solve the problem that existing visible and infrared image fusion methods rely only on the original local or global information representation, which has the problem of edge blurring and non-protrusion of salient targets, this paper proposes a layered fusion method based on channel attention mechanism and improved Generative Adversarial Network (HFCA_GAN). Firstly, the infrared image and visible image are decomposed into a base layer and fine layer, respectively, by a guiding filter. Secondly, the visible light base layer is fused with the infrared image base layer by histogram mapping enhancement to improve the contour effect. Thirdly, the improved GAN algorithm is used to fuse the infrared and visible image refinement layer, and the depth transferable module and guided fusion network are added to enrich the detailed information of the fused image. Finally, the multilayer convolutional fusion network with channel attention mechanism is used to correlate the local information of the layered fusion image, and the final fusion image containing contour gradient information and useful details is obtained. TNO and RoadSence datasets are selected for training and testing. The results show that the proposed algorithm retains the global structure features of multilayer images and has obvious advantages in fusion performance, model generalization and computational efficiency.https://www.mdpi.com/1424-8220/24/21/6916image fusionguided filtergenerative adversarial networkhistogram mappingchannel attention mechanism |
| spellingShingle | Jie Wu Shuai Yang Xiaoming Wang Yu Pei Shuai Wang Congcong Song Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks Sensors image fusion guided filter generative adversarial network histogram mapping channel attention mechanism |
| title | Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks |
| title_full | Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks |
| title_fullStr | Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks |
| title_full_unstemmed | Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks |
| title_short | Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks |
| title_sort | hierarchical fusion of infrared and visible images based on channel attention mechanism and generative adversarial networks |
| topic | image fusion guided filter generative adversarial network histogram mapping channel attention mechanism |
| url | https://www.mdpi.com/1424-8220/24/21/6916 |
| work_keys_str_mv | AT jiewu hierarchicalfusionofinfraredandvisibleimagesbasedonchannelattentionmechanismandgenerativeadversarialnetworks AT shuaiyang hierarchicalfusionofinfraredandvisibleimagesbasedonchannelattentionmechanismandgenerativeadversarialnetworks AT xiaomingwang hierarchicalfusionofinfraredandvisibleimagesbasedonchannelattentionmechanismandgenerativeadversarialnetworks AT yupei hierarchicalfusionofinfraredandvisibleimagesbasedonchannelattentionmechanismandgenerativeadversarialnetworks AT shuaiwang hierarchicalfusionofinfraredandvisibleimagesbasedonchannelattentionmechanismandgenerativeadversarialnetworks AT congcongsong hierarchicalfusionofinfraredandvisibleimagesbasedonchannelattentionmechanismandgenerativeadversarialnetworks |