Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning
In the context of multiple autonomous undersea vehicles(AUVs) executing underwater target interception missions, AUVs are required to make precise decisions based on both enemy and partner information, navigating the dual challenges of competition and cooperation. Most existing research typically fo...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Science Press (China)
2025-04-01
|
| Series: | 水下无人系统学报 |
| Subjects: | |
| Online Access: | https://sxwrxtxb.xml-journal.net/cn/article/doi/10.11993/j.issn.2096-3920.2025-0004 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849428773263376384 |
|---|---|
| author | Wenhao GAN Yunfei PENG Lei QIAO |
| author_facet | Wenhao GAN Yunfei PENG Lei QIAO |
| author_sort | Wenhao GAN |
| collection | DOAJ |
| description | In the context of multiple autonomous undersea vehicles(AUVs) executing underwater target interception missions, AUVs are required to make precise decisions based on both enemy and partner information, navigating the dual challenges of competition and cooperation. Most existing research typically focuses on single-target interception in simple environments and lacks a detailed exploration of collaborative mechanisms for multi-target interception mechanisms in complex environments. Therefore, this paper proposed a multi-agent deep reinforcement learning framework for AUVs to learn interception strategies in environments with complex obstacles and time-vary ocean currents, with a focus on cooperation in many-to-many game scenarios. First, a hierarchical maneuvering framework was introduced to improve the decision-making ability of AUVs through a three-layer loop structure. Next, the multi-agent proximal policy optimization algorithm was used to construct a scalable state and action space and design a compound reward function, enhancing interception efficiency and cooperation of AUVs. Finally, a population expansion–curriculum learning approach was incorporated within a centralized training and distributed execution architecture to help AUVs master generalizable cooperation strategies. Training results show rapid convergence and high success rates of the proposed interception strategies. The simulation experiments show that the trained AUVs can use the same set of models in multiple population configurations to effectively intercept multiple intruding targets through cooperation while avoiding obstacles. |
| format | Article |
| id | doaj-art-903888ff60e64aeebc12f5ccb1ce99af |
| institution | Kabale University |
| issn | 2096-3920 |
| language | zho |
| publishDate | 2025-04-01 |
| publisher | Science Press (China) |
| record_format | Article |
| series | 水下无人系统学报 |
| spelling | doaj-art-903888ff60e64aeebc12f5ccb1ce99af2025-08-20T03:28:34ZzhoScience Press (China)水下无人系统学报2096-39202025-04-0133232533210.11993/j.issn.2096-3920.2025-00042025-0004Multi-Underwater Target Interception Strategy Based on Deep Reinforcement LearningWenhao GAN0Yunfei PENG1Lei QIAO2School of Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaSchool of Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaSchool of Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaIn the context of multiple autonomous undersea vehicles(AUVs) executing underwater target interception missions, AUVs are required to make precise decisions based on both enemy and partner information, navigating the dual challenges of competition and cooperation. Most existing research typically focuses on single-target interception in simple environments and lacks a detailed exploration of collaborative mechanisms for multi-target interception mechanisms in complex environments. Therefore, this paper proposed a multi-agent deep reinforcement learning framework for AUVs to learn interception strategies in environments with complex obstacles and time-vary ocean currents, with a focus on cooperation in many-to-many game scenarios. First, a hierarchical maneuvering framework was introduced to improve the decision-making ability of AUVs through a three-layer loop structure. Next, the multi-agent proximal policy optimization algorithm was used to construct a scalable state and action space and design a compound reward function, enhancing interception efficiency and cooperation of AUVs. Finally, a population expansion–curriculum learning approach was incorporated within a centralized training and distributed execution architecture to help AUVs master generalizable cooperation strategies. Training results show rapid convergence and high success rates of the proposed interception strategies. The simulation experiments show that the trained AUVs can use the same set of models in multiple population configurations to effectively intercept multiple intruding targets through cooperation while avoiding obstacles.https://sxwrxtxb.xml-journal.net/cn/article/doi/10.11993/j.issn.2096-3920.2025-0004autonomous undersea vehiclemulti-agentreinforcement learningtarget interceptionintelligent decision-making |
| spellingShingle | Wenhao GAN Yunfei PENG Lei QIAO Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning 水下无人系统学报 autonomous undersea vehicle multi-agent reinforcement learning target interception intelligent decision-making |
| title | Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning |
| title_full | Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning |
| title_fullStr | Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning |
| title_full_unstemmed | Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning |
| title_short | Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning |
| title_sort | multi underwater target interception strategy based on deep reinforcement learning |
| topic | autonomous undersea vehicle multi-agent reinforcement learning target interception intelligent decision-making |
| url | https://sxwrxtxb.xml-journal.net/cn/article/doi/10.11993/j.issn.2096-3920.2025-0004 |
| work_keys_str_mv | AT wenhaogan multiunderwatertargetinterceptionstrategybasedondeepreinforcementlearning AT yunfeipeng multiunderwatertargetinterceptionstrategybasedondeepreinforcementlearning AT leiqiao multiunderwatertargetinterceptionstrategybasedondeepreinforcementlearning |