IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm
Multi-object tracking aims to track multiple objects across consecutive frames in a video, assigning a unique classifier to each object. However, issues such as occlusions, directional changes, or shape alterations can cause appearance variations, leading to detection and matching problems that in t...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/1/107 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549459233701888 |
|---|---|
| author | Hongbin Liu Yongze Zhao Peng Dong Xiuyi Guo Yilin Wang |
| author_facet | Hongbin Liu Yongze Zhao Peng Dong Xiuyi Guo Yilin Wang |
| author_sort | Hongbin Liu |
| collection | DOAJ |
| description | Multi-object tracking aims to track multiple objects across consecutive frames in a video, assigning a unique classifier to each object. However, issues such as occlusions, directional changes, or shape alterations can cause appearance variations, leading to detection and matching problems that in turn result in frequent ID switches. To solve these issues, this paper proposes a two-stage multi-object tracking framework based on a spatial and temporal fusion algorithm. First, the video frames are processed by a detector to identify objects and form rectangular detection areas. Meanwhile, an estimator predicts the target rectangular areas in the next frame. Then, we extract the optical flow of the target pixels within the detection and prediction areas, and then a temporal information model is established by calculating the average of the target pixels’ optical flow. Afterward, we present a spatial information model using the R-IoU (Reverse of Intersection over Union) between the detection and prediction areas. This spatial and temporal information is combined with weighted matrix fusion, which achieves the feature matching and association task. Finally, we implement a two-stage association multi-object tracking model using the mentioned fusion algorithm. Experiments on the MOTChallenge dataset using the official detector show that our two-stage multi-object tracking method based on the spatial and temporal fusion algorithm is robust in handling occlusions and ID switch issues. As of the submission of this paper, the proposed method has achieved the top ranking in the MOT17 benchmark when evaluated with the official detector. |
| format | Article |
| id | doaj-art-a085c5bdcf4841708937f8dd15e02791 |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-a085c5bdcf4841708937f8dd15e027912025-01-10T13:14:28ZengMDPI AGApplied Sciences2076-34172024-12-0115110710.3390/app15010107IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion AlgorithmHongbin Liu0Yongze Zhao1Peng Dong2Xiuyi Guo3Yilin Wang4Shandong Key Laboratory of Smart Buildings and Energy Efficiency, School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaShandong Key Laboratory of Smart Buildings and Energy Efficiency, School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaShandong Key Laboratory of Smart Buildings and Energy Efficiency, School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaShandong Key Laboratory of Smart Buildings and Energy Efficiency, School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaShandong Key Laboratory of Smart Buildings and Energy Efficiency, School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaMulti-object tracking aims to track multiple objects across consecutive frames in a video, assigning a unique classifier to each object. However, issues such as occlusions, directional changes, or shape alterations can cause appearance variations, leading to detection and matching problems that in turn result in frequent ID switches. To solve these issues, this paper proposes a two-stage multi-object tracking framework based on a spatial and temporal fusion algorithm. First, the video frames are processed by a detector to identify objects and form rectangular detection areas. Meanwhile, an estimator predicts the target rectangular areas in the next frame. Then, we extract the optical flow of the target pixels within the detection and prediction areas, and then a temporal information model is established by calculating the average of the target pixels’ optical flow. Afterward, we present a spatial information model using the R-IoU (Reverse of Intersection over Union) between the detection and prediction areas. This spatial and temporal information is combined with weighted matrix fusion, which achieves the feature matching and association task. Finally, we implement a two-stage association multi-object tracking model using the mentioned fusion algorithm. Experiments on the MOTChallenge dataset using the official detector show that our two-stage multi-object tracking method based on the spatial and temporal fusion algorithm is robust in handling occlusions and ID switch issues. As of the submission of this paper, the proposed method has achieved the top ranking in the MOT17 benchmark when evaluated with the official detector.https://www.mdpi.com/2076-3417/15/1/107multi-object trackingID switchesfusion algorithmspatial and temporal information |
| spellingShingle | Hongbin Liu Yongze Zhao Peng Dong Xiuyi Guo Yilin Wang IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm Applied Sciences multi-object tracking ID switches fusion algorithm spatial and temporal information |
| title | IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm |
| title_full | IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm |
| title_fullStr | IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm |
| title_full_unstemmed | IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm |
| title_short | IOF-Tracker: A Two-Stage Multiple Targets Tracking Method Using Spatial-Temporal Fusion Algorithm |
| title_sort | iof tracker a two stage multiple targets tracking method using spatial temporal fusion algorithm |
| topic | multi-object tracking ID switches fusion algorithm spatial and temporal information |
| url | https://www.mdpi.com/2076-3417/15/1/107 |
| work_keys_str_mv | AT hongbinliu ioftrackeratwostagemultipletargetstrackingmethodusingspatialtemporalfusionalgorithm AT yongzezhao ioftrackeratwostagemultipletargetstrackingmethodusingspatialtemporalfusionalgorithm AT pengdong ioftrackeratwostagemultipletargetstrackingmethodusingspatialtemporalfusionalgorithm AT xiuyiguo ioftrackeratwostagemultipletargetstrackingmethodusingspatialtemporalfusionalgorithm AT yilinwang ioftrackeratwostagemultipletargetstrackingmethodusingspatialtemporalfusionalgorithm |