Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection
Limited by the imaging capabilities of sensors, research based on single modality is difficult to cope with faults and dynamic perturbations in detection. Effective multispectral object detection, which can achieve better detection accuracy by fusing visual information from different modalities, has...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/21/4034 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850062863113125888 |
|---|---|
| author | Runze Guo Xiaojun Guo Xiaoyong Sun Peida Zhou Bei Sun Shaojing Su |
| author_facet | Runze Guo Xiaojun Guo Xiaoyong Sun Peida Zhou Bei Sun Shaojing Su |
| author_sort | Runze Guo |
| collection | DOAJ |
| description | Limited by the imaging capabilities of sensors, research based on single modality is difficult to cope with faults and dynamic perturbations in detection. Effective multispectral object detection, which can achieve better detection accuracy by fusing visual information from different modalities, has attracted widespread attention. However, most of the existing methods adopt simple fusion mechanisms, which fail to utilize the complementary information between modalities while lacking the guidance of a priori knowledge. To address the above issues, we propose a novel background-aware cross-attention multiscale fusion network (BA-CAMF Net) to achieve adaptive fusion in visible and infrared images. First, a background-aware module is designed to calculate the light and contrast to guide the fusion. Then, a cross-attention multiscale fusion module is put forward to enhance inter-modality complement features and intra-modality intrinsic features. Finally, multiscale feature maps from different modalities are fused according to background-aware weights. Experimental results on LLVIP, FLIR, and VEDAI indicate that the proposed BA-CAMF Net achieves higher detection accuracy than the current State-of-the-Art multispectral detectors. |
| format | Article |
| id | doaj-art-5ccc75389bc64ff0833d591e67866145 |
| institution | DOAJ |
| issn | 2072-4292 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-5ccc75389bc64ff0833d591e678661452025-08-20T02:49:49ZengMDPI AGRemote Sensing2072-42922024-10-011621403410.3390/rs16214034Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object DetectionRunze Guo0Xiaojun Guo1Xiaoyong Sun2Peida Zhou3Bei Sun4Shaojing Su5College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaLimited by the imaging capabilities of sensors, research based on single modality is difficult to cope with faults and dynamic perturbations in detection. Effective multispectral object detection, which can achieve better detection accuracy by fusing visual information from different modalities, has attracted widespread attention. However, most of the existing methods adopt simple fusion mechanisms, which fail to utilize the complementary information between modalities while lacking the guidance of a priori knowledge. To address the above issues, we propose a novel background-aware cross-attention multiscale fusion network (BA-CAMF Net) to achieve adaptive fusion in visible and infrared images. First, a background-aware module is designed to calculate the light and contrast to guide the fusion. Then, a cross-attention multiscale fusion module is put forward to enhance inter-modality complement features and intra-modality intrinsic features. Finally, multiscale feature maps from different modalities are fused according to background-aware weights. Experimental results on LLVIP, FLIR, and VEDAI indicate that the proposed BA-CAMF Net achieves higher detection accuracy than the current State-of-the-Art multispectral detectors.https://www.mdpi.com/2072-4292/16/21/4034multispectral object detectioncomplementary informationpriori knowledgebackground awarecross attentionmultiscale fusion |
| spellingShingle | Runze Guo Xiaojun Guo Xiaoyong Sun Peida Zhou Bei Sun Shaojing Su Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection Remote Sensing multispectral object detection complementary information priori knowledge background aware cross attention multiscale fusion |
| title | Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection |
| title_full | Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection |
| title_fullStr | Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection |
| title_full_unstemmed | Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection |
| title_short | Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection |
| title_sort | background aware cross attention multiscale fusion for multispectral object detection |
| topic | multispectral object detection complementary information priori knowledge background aware cross attention multiscale fusion |
| url | https://www.mdpi.com/2072-4292/16/21/4034 |
| work_keys_str_mv | AT runzeguo backgroundawarecrossattentionmultiscalefusionformultispectralobjectdetection AT xiaojunguo backgroundawarecrossattentionmultiscalefusionformultispectralobjectdetection AT xiaoyongsun backgroundawarecrossattentionmultiscalefusionformultispectralobjectdetection AT peidazhou backgroundawarecrossattentionmultiscalefusionformultispectralobjectdetection AT beisun backgroundawarecrossattentionmultiscalefusionformultispectralobjectdetection AT shaojingsu backgroundawarecrossattentionmultiscalefusionformultispectralobjectdetection |