Visible-infrared person re-identification with region-based augmentation and cross modality attention
Abstract Visible-infrared person re-identification (VI-ReID) aims to search the same pedestrian of interest across visible and infrared modalities. Existing models mainly focus on compensating for modality-specific information to reduce modality variation. However, these methods often introduce inte...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-05-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-01979-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Abstract Visible-infrared person re-identification (VI-ReID) aims to search the same pedestrian of interest across visible and infrared modalities. Existing models mainly focus on compensating for modality-specific information to reduce modality variation. However, these methods often introduce interfering information and lead to higher computational overhead when generating the corresponding images or features. Additionally, the pedestrian region characteristics in VI-ReID are not effectively utilized, thus resulting in ambiguous or unnatural images. To address these issues, it is critical to leverage pedestrian attentive features and learn modality-complete and -consistent representation. In this paper, a novel Region-based Augmentation and Cross Modality Attention (RACA) model is proposed, focusing on the pedestrian regions to efficiently compensate for missing modality-specific features. Specifically, we propose a region-based data augmentation module PedMix to enhance pedestrian region coherence by mixing the corresponding regions from different modalities, thus generating more natural images. Moreover, a lightweight hybrid compensation module, i.e., a Modality Feature Transfer (MFT) module, is proposed to integrate cross attention and convolution networks to avoid introducing interfering information while preserving minimal computational overhead. Extensive experiments conducted on the benchmark SYSU-MM01 and RegDB datasets demonstrated the effectiveness of our proposed RACA model. |
|---|---|
| ISSN: | 2045-2322 |