YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features
With the rapid advancements in deep learning technology, various deep learning-based object detection algorithms have found extensive applications in UAV-related tasks. However, motivated by the fact that current object detection algorithms for unimodal aerial remote sensing images fail to achieve a...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10643643/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850023026497683456 |
|---|---|
| author | Huiying Wang Chunping Wang Qiang Fu Binqiang Si Dongdong Zhang Renke Kou Ying Yu Changfeng Feng |
| author_facet | Huiying Wang Chunping Wang Qiang Fu Binqiang Si Dongdong Zhang Renke Kou Ying Yu Changfeng Feng |
| author_sort | Huiying Wang |
| collection | DOAJ |
| description | With the rapid advancements in deep learning technology, various deep learning-based object detection algorithms have found extensive applications in UAV-related tasks. However, motivated by the fact that current object detection algorithms for unimodal aerial remote sensing images fail to achieve around-the-clock object detection. To tackle this, we propose an around-the-clock object detection algorithm YOLOFIV that fuses infrared and visible features. First, we design a dual-stream backbone network based on the attention mechanism to adequately extract the features of both modalities. Moreover, the ECA attention mechanism is integrated into the feature enhancement network to amplify attention toward challenging detection scenarios. Finally, we improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method YOLOFIV on the widely used drone vehicle dataset, YOLOFIV achieves an accuracy of 64.71% (in terms of <inline-formula><tex-math notation="LaTeX">$\text{mean average precision}_{0.5}$</tex-math></inline-formula>), accuracy improvement of 8.32% over baseline bimodal model, similar performance to UACMD designed for ARSI object detection but with 92.35% reduction in parameter count, and 17.87 times speedup. The results show that our approach achieves round-the-clock object detection while maintaining a favorable accuracy-speed tradeoff. |
| format | Article |
| id | doaj-art-2cf0880d0fb44bc5bb3c474eb76085e9 |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-2cf0880d0fb44bc5bb3c474eb76085e92025-08-20T03:01:30ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352024-01-0117152691528710.1109/JSTARS.2024.344764910643643YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible FeaturesHuiying Wang0https://orcid.org/0009-0004-0346-3585Chunping Wang1Qiang Fu2https://orcid.org/0000-0002-3831-9856Binqiang Si3https://orcid.org/0000-0003-3206-1388Dongdong Zhang4https://orcid.org/0000-0003-3817-5706Renke Kou5https://orcid.org/0000-0001-5893-3127Ying Yu6https://orcid.org/0000-0001-7840-9891Changfeng Feng7https://orcid.org/0000-0003-4413-1611Army Engineering University of PLA, Shijiazhuang, ChinaArmy Engineering University of PLA, Shijiazhuang, ChinaArmy Engineering University of PLA, Shijiazhuang, ChinaSchool of Instrument Science and Opto-electronics Engineering, Beijing Information Science and Technology University, Beijing, ChinaArmy Engineering University of PLA, Shijiazhuang, ChinaArmy Engineering University of PLA, Shijiazhuang, ChinaArmy Engineering University of PLA, Shijiazhuang, ChinaArmy Engineering University of PLA, Shijiazhuang, ChinaWith the rapid advancements in deep learning technology, various deep learning-based object detection algorithms have found extensive applications in UAV-related tasks. However, motivated by the fact that current object detection algorithms for unimodal aerial remote sensing images fail to achieve around-the-clock object detection. To tackle this, we propose an around-the-clock object detection algorithm YOLOFIV that fuses infrared and visible features. First, we design a dual-stream backbone network based on the attention mechanism to adequately extract the features of both modalities. Moreover, the ECA attention mechanism is integrated into the feature enhancement network to amplify attention toward challenging detection scenarios. Finally, we improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method YOLOFIV on the widely used drone vehicle dataset, YOLOFIV achieves an accuracy of 64.71% (in terms of <inline-formula><tex-math notation="LaTeX">$\text{mean average precision}_{0.5}$</tex-math></inline-formula>), accuracy improvement of 8.32% over baseline bimodal model, similar performance to UACMD designed for ARSI object detection but with 92.35% reduction in parameter count, and 17.87 times speedup. The results show that our approach achieves round-the-clock object detection while maintaining a favorable accuracy-speed tradeoff.https://ieeexplore.ieee.org/document/10643643/Attention mechanismfeature fusionmultimodal aerial remote sensing images (ARSIs)object detectionround-the-clock detection |
| spellingShingle | Huiying Wang Chunping Wang Qiang Fu Binqiang Si Dongdong Zhang Renke Kou Ying Yu Changfeng Feng YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Attention mechanism feature fusion multimodal aerial remote sensing images (ARSIs) object detection round-the-clock detection |
| title | YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features |
| title_full | YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features |
| title_fullStr | YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features |
| title_full_unstemmed | YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features |
| title_short | YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features |
| title_sort | yolofiv object detection algorithm for around the clock aerial remote sensing images by fusing infrared and visible features |
| topic | Attention mechanism feature fusion multimodal aerial remote sensing images (ARSIs) object detection round-the-clock detection |
| url | https://ieeexplore.ieee.org/document/10643643/ |
| work_keys_str_mv | AT huiyingwang yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT chunpingwang yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT qiangfu yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT binqiangsi yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT dongdongzhang yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT renkekou yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT yingyu yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures AT changfengfeng yolofivobjectdetectionalgorithmforaroundtheclockaerialremotesensingimagesbyfusinginfraredandvisiblefeatures |