Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking
Recently, there has been growing interest in the development of 3D multi-view, multi-object detection and tracking models (MV-MOD and MV-MOT), resulting in significant methodological advances. However, many of these developments do not address the critical challenge of generalization across differen...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2025-07-01
|
| Series: | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-annals.copernicus.org/articles/X-G-2025/47/2025/isprs-annals-X-G-2025-47-2025.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849321207519772672 |
|---|---|
| author | R. Ali M. Mehltretter C. Heipke |
| author_facet | R. Ali M. Mehltretter C. Heipke |
| author_sort | R. Ali |
| collection | DOAJ |
| description | Recently, there has been growing interest in the development of 3D multi-view, multi-object detection and tracking models (MV-MOD and MV-MOT), resulting in significant methodological advances. However, many of these developments do not address the critical challenge of generalization across different camera constellations, i.e., having camera constellations that differ between training and testing, limiting their effectiveness in real-world applications. A key factor often overlooked is the influence of the direction of the optical axis during image capture, which is not adequately propagated in the model. In this work, we propose a novel convolutional neural network-based method for 3D MV-MOD and MV-MOT that enhances generalization by incorporating the direction from which the images were captured as an additional input to this network. For each image, this directional information is combined with the 2D features extracted from that image, before 3D features are computed, using the 2D features from all images. We empirically evaluate the performance of the proposed method on the real-world Wildtrack dataset, demonstrating the effectiveness of the proposed approach. |
| format | Article |
| id | doaj-art-ef7e6f57d52a402c85f07a8403f65638 |
| institution | Kabale University |
| issn | 2194-9042 2194-9050 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj-art-ef7e6f57d52a402c85f07a8403f656382025-08-20T03:49:49ZengCopernicus PublicationsISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences2194-90422194-90502025-07-01X-G-2025475510.5194/isprs-annals-X-G-2025-47-2025Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian TrackingR. Ali0M. Mehltretter1C. Heipke2Institute of Photogrammetry and GeoInformation, Leibniz University Hannover, GermanyInstitute of Photogrammetry and GeoInformation, Leibniz University Hannover, GermanyInstitute of Photogrammetry and GeoInformation, Leibniz University Hannover, GermanyRecently, there has been growing interest in the development of 3D multi-view, multi-object detection and tracking models (MV-MOD and MV-MOT), resulting in significant methodological advances. However, many of these developments do not address the critical challenge of generalization across different camera constellations, i.e., having camera constellations that differ between training and testing, limiting their effectiveness in real-world applications. A key factor often overlooked is the influence of the direction of the optical axis during image capture, which is not adequately propagated in the model. In this work, we propose a novel convolutional neural network-based method for 3D MV-MOD and MV-MOT that enhances generalization by incorporating the direction from which the images were captured as an additional input to this network. For each image, this directional information is combined with the 2D features extracted from that image, before 3D features are computed, using the 2D features from all images. We empirically evaluate the performance of the proposed method on the real-world Wildtrack dataset, demonstrating the effectiveness of the proposed approach.https://isprs-annals.copernicus.org/articles/X-G-2025/47/2025/isprs-annals-X-G-2025-47-2025.pdf |
| spellingShingle | R. Ali M. Mehltretter C. Heipke Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking |
| title_full | Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking |
| title_fullStr | Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking |
| title_full_unstemmed | Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking |
| title_short | Integrating Viewing Direction and Image Features for Robust Multi-View Multi-Object 3D Pedestrian Tracking |
| title_sort | integrating viewing direction and image features for robust multi view multi object 3d pedestrian tracking |
| url | https://isprs-annals.copernicus.org/articles/X-G-2025/47/2025/isprs-annals-X-G-2025-47-2025.pdf |
| work_keys_str_mv | AT rali integratingviewingdirectionandimagefeaturesforrobustmultiviewmultiobject3dpedestriantracking AT mmehltretter integratingviewingdirectionandimagefeaturesforrobustmultiviewmultiobject3dpedestriantracking AT cheipke integratingviewingdirectionandimagefeaturesforrobustmultiviewmultiobject3dpedestriantracking |