Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios
Environment sensing plays an important role for the safe autonomous navigation of intelligent ships. However, the inherent limitations of sensors, such as the low frequency of the automatic identification system (AIS), blind zone of the marine radar, and lack of depth information in visible images,...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/7/2179 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849730241914732544 |
|---|---|
| author | Wenbin Huang Hui Feng Haixiang Xu Xu Liu Jianhua He Langxiong Gan Xiaoqian Wang Shanshan Wang |
| author_facet | Wenbin Huang Hui Feng Haixiang Xu Xu Liu Jianhua He Langxiong Gan Xiaoqian Wang Shanshan Wang |
| author_sort | Wenbin Huang |
| collection | DOAJ |
| description | Environment sensing plays an important role for the safe autonomous navigation of intelligent ships. However, the inherent limitations of sensors, such as the low frequency of the automatic identification system (AIS), blind zone of the marine radar, and lack of depth information in visible images, make it difficult to achieve accurate sensing with a single modality of sensor data. To overcome this limitation, we propose a new multi-source data fusion framework and technologies that integrate AIS, radar, and visible data. This framework leverages the complementary strengths of these different types of sensors to enhance sensing performance, especially in real complex scenarios where single-modality data are significantly affected by blind zone and adverse weather conditions. We first design a multi-stage detection and tracking method (named MSTrack). By feeding the historical fusion results back to earlier tracking stages, the proposed method identifies and refines potential missing detections from the layered detection and tracking processes of radar and visible images. Then, a cascade association matching method is proposed to realize the association between multi-source trajectories. It first performs pairwise association in a high-accuracy aligned coordinate system, followed by association in a low-accuracy coordinate system and integrated matching between multi-source data. Through these association operations, the method can effectively reduce the association errors caused by measurement noise and projection system errors. Furthermore, we develop the first multi-source fusion dataset for intelligent vessel (WHUT-MSFVessel), and validate our methods. The experimental results show that our multi-source data fusion methods significantly improve the sensing accuracy and identity consistency of tracking, achieving average <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>M</mi><mi>O</mi><mi>T</mi><mi>A</mi></mrow></semantics></math></inline-formula> scores of 0.872 and 0.938 on the radar and visible images, respectively, and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>I</mi><mi>D</mi><msub><mi>F</mi><mn>1</mn></msub></mrow></semantics></math></inline-formula> scores of 0.811 and 0.929. Additionally, the fusion accuracy reaches up to 0.9, which can provide vessels with a comprehensive perception of the navigation environment for safer navigation. |
| format | Article |
| id | doaj-art-31245c5f70f541a986731b2e3bfafa3a |
| institution | DOAJ |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-31245c5f70f541a986731b2e3bfafa3a2025-08-20T03:08:56ZengMDPI AGSensors1424-82202025-03-01257217910.3390/s25072179Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex ScenariosWenbin Huang0Hui Feng1Haixiang Xu2Xu Liu3Jianhua He4Langxiong Gan5Xiaoqian Wang6Shanshan Wang7Key Laboratory of High Performance Ship Technology (Wuhan University of Technology), Ministry of Education, Wuhan 430063, ChinaKey Laboratory of High Performance Ship Technology (Wuhan University of Technology), Ministry of Education, Wuhan 430063, ChinaKey Laboratory of High Performance Ship Technology (Wuhan University of Technology), Ministry of Education, Wuhan 430063, ChinaKey Laboratory of High Performance Ship Technology (Wuhan University of Technology), Ministry of Education, Wuhan 430063, ChinaSchool of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UKSchool of Navigation, Wuhan University of Technology, Wuhan 430063, ChinaKey Laboratory of High Performance Ship Technology (Wuhan University of Technology), Ministry of Education, Wuhan 430063, ChinaKey Laboratory of High Performance Ship Technology (Wuhan University of Technology), Ministry of Education, Wuhan 430063, ChinaEnvironment sensing plays an important role for the safe autonomous navigation of intelligent ships. However, the inherent limitations of sensors, such as the low frequency of the automatic identification system (AIS), blind zone of the marine radar, and lack of depth information in visible images, make it difficult to achieve accurate sensing with a single modality of sensor data. To overcome this limitation, we propose a new multi-source data fusion framework and technologies that integrate AIS, radar, and visible data. This framework leverages the complementary strengths of these different types of sensors to enhance sensing performance, especially in real complex scenarios where single-modality data are significantly affected by blind zone and adverse weather conditions. We first design a multi-stage detection and tracking method (named MSTrack). By feeding the historical fusion results back to earlier tracking stages, the proposed method identifies and refines potential missing detections from the layered detection and tracking processes of radar and visible images. Then, a cascade association matching method is proposed to realize the association between multi-source trajectories. It first performs pairwise association in a high-accuracy aligned coordinate system, followed by association in a low-accuracy coordinate system and integrated matching between multi-source data. Through these association operations, the method can effectively reduce the association errors caused by measurement noise and projection system errors. Furthermore, we develop the first multi-source fusion dataset for intelligent vessel (WHUT-MSFVessel), and validate our methods. The experimental results show that our multi-source data fusion methods significantly improve the sensing accuracy and identity consistency of tracking, achieving average <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>M</mi><mi>O</mi><mi>T</mi><mi>A</mi></mrow></semantics></math></inline-formula> scores of 0.872 and 0.938 on the radar and visible images, respectively, and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>I</mi><mi>D</mi><msub><mi>F</mi><mn>1</mn></msub></mrow></semantics></math></inline-formula> scores of 0.811 and 0.929. Additionally, the fusion accuracy reaches up to 0.9, which can provide vessels with a comprehensive perception of the navigation environment for safer navigation.https://www.mdpi.com/1424-8220/25/7/2179intelligent shipsnavigation safetyvessel detection and trackingmulti-source sensorsdata fusion |
| spellingShingle | Wenbin Huang Hui Feng Haixiang Xu Xu Liu Jianhua He Langxiong Gan Xiaoqian Wang Shanshan Wang Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios Sensors intelligent ships navigation safety vessel detection and tracking multi-source sensors data fusion |
| title | Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios |
| title_full | Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios |
| title_fullStr | Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios |
| title_full_unstemmed | Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios |
| title_short | Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex Scenarios |
| title_sort | surface vessels detection and tracking method and datasets with multi source data fusion in real world complex scenarios |
| topic | intelligent ships navigation safety vessel detection and tracking multi-source sensors data fusion |
| url | https://www.mdpi.com/1424-8220/25/7/2179 |
| work_keys_str_mv | AT wenbinhuang surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT huifeng surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT haixiangxu surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT xuliu surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT jianhuahe surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT langxionggan surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT xiaoqianwang surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios AT shanshanwang surfacevesselsdetectionandtrackingmethodanddatasetswithmultisourcedatafusioninrealworldcomplexscenarios |