Multispectral pedestrian detection based on feature complementation and enhancement
Abstract Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integra...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-11-01
|
| Series: | IET Intelligent Transport Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/itr2.12562 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850197610769416192 |
|---|---|
| author | Linzhen Nie Meihe Lu Zhiwei He Jiachen Hu Zhishuai Yin |
| author_facet | Linzhen Nie Meihe Lu Zhiwei He Jiachen Hu Zhishuai Yin |
| author_sort | Linzhen Nie |
| collection | DOAJ |
| description | Abstract Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integrates saliency detection as a sub‐task and utilizes global information for enhanced feature representation. The approach enhances thermal inputs by incorporating gradients at the raw‐data level before feature extraction. Utilizing a dual‐stream backbone, a global semantic information extraction module is introduced that combines pooling with horizontal–vertical attention mechanisms, capturing high‐quality global semantic information for lower‐level feature enrichment and guidance. Additionally, the pedestrian locality enhancement module is designed to enhance spatial locality information of pedestrians through saliency detection. Furthermore, to alleviate the challenges posed by positional shifts between cross‐spectral features, deformable convolution is innovatively employed. Experimental results on the KAIST dataset demonstrate that FCE‐RCNN significantly improves nighttime detection, achieving a log‐average miss rate of 6.92%, outperforming the new method ICAFusion by 0.93%. These results underscore the effectiveness of FCE‐RCNN, and the method also maintains competitive inference speed, making it suitable for real‐time applications. |
| format | Article |
| id | doaj-art-e7875880a4c5433d9245327eda3c2784 |
| institution | OA Journals |
| issn | 1751-956X 1751-9578 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Intelligent Transport Systems |
| spelling | doaj-art-e7875880a4c5433d9245327eda3c27842025-08-20T02:13:06ZengWileyIET Intelligent Transport Systems1751-956X1751-95782024-11-0118112166217710.1049/itr2.12562Multispectral pedestrian detection based on feature complementation and enhancementLinzhen Nie0Meihe Lu1Zhiwei He2Jiachen Hu3Zhishuai Yin4School of Automotive Engineering Wuhan University of Technology Wuhan ChinaSchool of Automotive Engineering Wuhan University of Technology Wuhan ChinaSchool of Automotive Engineering Wuhan University of Technology Wuhan ChinaSchool of Automotive Engineering Wuhan University of Technology Wuhan ChinaSchool of Automotive Engineering Wuhan University of Technology Wuhan ChinaAbstract Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integrates saliency detection as a sub‐task and utilizes global information for enhanced feature representation. The approach enhances thermal inputs by incorporating gradients at the raw‐data level before feature extraction. Utilizing a dual‐stream backbone, a global semantic information extraction module is introduced that combines pooling with horizontal–vertical attention mechanisms, capturing high‐quality global semantic information for lower‐level feature enrichment and guidance. Additionally, the pedestrian locality enhancement module is designed to enhance spatial locality information of pedestrians through saliency detection. Furthermore, to alleviate the challenges posed by positional shifts between cross‐spectral features, deformable convolution is innovatively employed. Experimental results on the KAIST dataset demonstrate that FCE‐RCNN significantly improves nighttime detection, achieving a log‐average miss rate of 6.92%, outperforming the new method ICAFusion by 0.93%. These results underscore the effectiveness of FCE‐RCNN, and the method also maintains competitive inference speed, making it suitable for real‐time applications.https://doi.org/10.1049/itr2.12562automated drivingintelligent vehiclescomputer visionimage fusioninfrared imaging |
| spellingShingle | Linzhen Nie Meihe Lu Zhiwei He Jiachen Hu Zhishuai Yin Multispectral pedestrian detection based on feature complementation and enhancement IET Intelligent Transport Systems automated driving intelligent vehicles computer vision image fusion infrared imaging |
| title | Multispectral pedestrian detection based on feature complementation and enhancement |
| title_full | Multispectral pedestrian detection based on feature complementation and enhancement |
| title_fullStr | Multispectral pedestrian detection based on feature complementation and enhancement |
| title_full_unstemmed | Multispectral pedestrian detection based on feature complementation and enhancement |
| title_short | Multispectral pedestrian detection based on feature complementation and enhancement |
| title_sort | multispectral pedestrian detection based on feature complementation and enhancement |
| topic | automated driving intelligent vehicles computer vision image fusion infrared imaging |
| url | https://doi.org/10.1049/itr2.12562 |
| work_keys_str_mv | AT linzhennie multispectralpedestriandetectionbasedonfeaturecomplementationandenhancement AT meihelu multispectralpedestriandetectionbasedonfeaturecomplementationandenhancement AT zhiweihe multispectralpedestriandetectionbasedonfeaturecomplementationandenhancement AT jiachenhu multispectralpedestriandetectionbasedonfeaturecomplementationandenhancement AT zhishuaiyin multispectralpedestriandetectionbasedonfeaturecomplementationandenhancement |