Missingness-aware prompting for modality-missing RGBT tracking
Abstract RGBT tracking has drawn great attention recently due to its ability to leverage enhancement and complementary information from the RGB and thermal infrared modalities. Nevertheless, RGBT tracking in real-world scenarios inevitably encounters heavy modality-missing challenges caused by subst...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2025-07-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s44443-025-00142-y |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Abstract RGBT tracking has drawn great attention recently due to its ability to leverage enhancement and complementary information from the RGB and thermal infrared modalities. Nevertheless, RGBT tracking in real-world scenarios inevitably encounters heavy modality-missing challenges caused by substantial environmental factors (such as device overheating, and frame skipping). Existing methods for RGBT tracking are built upon pre-processed missingness-free datasets and suffer significant performance degradation when applied to noisy datasets with random missing modalities. In this paper, we propose a novel missingness-aware prompting framework (MAP) for modality-missing RGBT tracking. It is a lightweight prompting framework consisting of two-stage prompts focusing on compensating essential information for RGBT tracking stage-by-stage. Specifically, prototypical missingness-aware prompts (pMAP) are explored to compensate for modality-specific but instance-agnostic prototypical missing information. Contextual missingness-aware prompts (cMAP) are further designed to compensate for instance-specific detailed missing information. Extensive experiments on three large-scale datasets demonstrate the effectiveness and superiority of the proposed framework for RGBT tracking with random missing modalities. |
|---|---|
| ISSN: | 1319-1578 2213-1248 |