DMCM: Dwo-branch multilevel feature fusion with cross-attention mechanism for infrared and visible image fusion.
In response to the limitations of current infrared and visible light image fusion algorithms-namely insufficient feature extraction, loss of detailed texture information, underutilization of differential and shared information, and the high number of model parameters-this paper proposes a novel mult...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0318931 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | In response to the limitations of current infrared and visible light image fusion algorithms-namely insufficient feature extraction, loss of detailed texture information, underutilization of differential and shared information, and the high number of model parameters-this paper proposes a novel multi-scale infrared and visible image fusion method with two-branch feature interaction. The proposed method introduces a lightweight multi-scale group convolution, based on GS convolution, which enhances multi-scale information interaction while reducing network parameters by incorporating group convolution and stacking multiple small convolutional kernels. Furthermore, the multi-level attention module is improved by integrating edge-enhanced branches and depthwise separable convolutions to preserve detailed texture information. Additionally, a lightweight cross-attention fusion module is introduced, optimizing the use of differential and shared features while minimizing computational complexity. Lastly, the efficiency of local attention is enhanced by adding a multi-dimensional fusion branch, which bolsters the interaction of information across multiple dimensions and facilitates comprehensive spatial information extraction from multimodal images. The proposed algorithm, along with seven others, was tested extensively on public datasets such as TNO and Roadscene. The experimental results demonstrate that the proposed method outperforms other algorithms in both subjective and objective evaluation results. Additionally, it demonstrates good performance in terms of operational efficiency. Moreover, target detection performance experiments conducted on the [Formula: see text] dataset confirm the superior performance of the proposed algorithm. |
|---|---|
| ISSN: | 1932-6203 |