Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment
To enable collaborative scouting / strike / assessment of underwater time-sensitive targets by heterogeneous multi-agent systems, in this study a heterogeneous multi-agent collaborative decision-making method is proposed based on deep reinforcement learning. The method integrates two core learning f...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10988794/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850153186421112832 |
|---|---|
| author | Shoumin Wang Ning Niu Zhichao Wang Yaxuan Lv Jing Zhang |
| author_facet | Shoumin Wang Ning Niu Zhichao Wang Yaxuan Lv Jing Zhang |
| author_sort | Shoumin Wang |
| collection | DOAJ |
| description | To enable collaborative scouting / strike / assessment of underwater time-sensitive targets by heterogeneous multi-agent systems, in this study a heterogeneous multi-agent collaborative decision-making method is proposed based on deep reinforcement learning. The method integrates two core learning frameworks one for heterogeneous multi-agent task allocation and one for single-agent multi-task learning. The task allocation framework combines the proximal policy optimization algorithm with experience replay to train an Actor network, ensuring stable iterative updates of task allocation policies toward high-reward directions. Concurrently, a Critic network is trained using successor features and temporal difference methods, establishing a foundation for transfer learning. A region partitioning mechanism is introduced to construct a foundational knowledge base, enabling the transfer of sub-region knowledge acquired by multi-agents to target regions, thereby addressing complex underwater task allocation scenarios. The single-agent multi-task learning framework employs an experience replay pool with knowledge transfer attributes and policy distillation technology, allowing each agent to assimilate task-specific expertise from heterogeneous peers across diverse mission scenarios. This capability supports multi-task execution, including path planning, emergency obstacle avoidance, and trajectory tracking. |
| format | Article |
| id | doaj-art-be1e01601c7b474ea0cdd5ad2aa71be9 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-be1e01601c7b474ea0cdd5ad2aa71be92025-08-20T02:25:47ZengIEEEIEEE Access2169-35362025-01-0113842028421610.1109/ACCESS.2025.356750310988794Heterogeneous Multi-Agent Task Planning Method in Complex Marine EnvironmentShoumin Wang0Ning Niu1https://orcid.org/0000-0002-6431-2279Zhichao Wang2https://orcid.org/0009-0007-5244-2301Yaxuan Lv3Jing Zhang4https://orcid.org/0000-0001-6361-6898School of Mechatronic Engineering, Xi’an Technological University, Xi’an, ChinaSchool of Electronic Science and Technology, Hainan University, Haikou, ChinaThe 32033 Troops of China People’s Liberation Army, Haikou, ChinaSchool of Electronic Science and Technology, Hainan University, Haikou, ChinaSchool of Electronic Science and Technology, Hainan University, Haikou, ChinaTo enable collaborative scouting / strike / assessment of underwater time-sensitive targets by heterogeneous multi-agent systems, in this study a heterogeneous multi-agent collaborative decision-making method is proposed based on deep reinforcement learning. The method integrates two core learning frameworks one for heterogeneous multi-agent task allocation and one for single-agent multi-task learning. The task allocation framework combines the proximal policy optimization algorithm with experience replay to train an Actor network, ensuring stable iterative updates of task allocation policies toward high-reward directions. Concurrently, a Critic network is trained using successor features and temporal difference methods, establishing a foundation for transfer learning. A region partitioning mechanism is introduced to construct a foundational knowledge base, enabling the transfer of sub-region knowledge acquired by multi-agents to target regions, thereby addressing complex underwater task allocation scenarios. The single-agent multi-task learning framework employs an experience replay pool with knowledge transfer attributes and policy distillation technology, allowing each agent to assimilate task-specific expertise from heterogeneous peers across diverse mission scenarios. This capability supports multi-task execution, including path planning, emergency obstacle avoidance, and trajectory tracking.https://ieeexplore.ieee.org/document/10988794/Autonomous underwater vehiclecollaboration decision-makingexperiences replay methodproximal policy optimization algorithmsuccessor featureuncrewed surface vessels (USVs) |
| spellingShingle | Shoumin Wang Ning Niu Zhichao Wang Yaxuan Lv Jing Zhang Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment IEEE Access Autonomous underwater vehicle collaboration decision-making experiences replay method proximal policy optimization algorithm successor feature uncrewed surface vessels (USVs) |
| title | Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment |
| title_full | Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment |
| title_fullStr | Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment |
| title_full_unstemmed | Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment |
| title_short | Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment |
| title_sort | heterogeneous multi agent task planning method in complex marine environment |
| topic | Autonomous underwater vehicle collaboration decision-making experiences replay method proximal policy optimization algorithm successor feature uncrewed surface vessels (USVs) |
| url | https://ieeexplore.ieee.org/document/10988794/ |
| work_keys_str_mv | AT shouminwang heterogeneousmultiagenttaskplanningmethodincomplexmarineenvironment AT ningniu heterogeneousmultiagenttaskplanningmethodincomplexmarineenvironment AT zhichaowang heterogeneousmultiagenttaskplanningmethodincomplexmarineenvironment AT yaxuanlv heterogeneousmultiagenttaskplanningmethodincomplexmarineenvironment AT jingzhang heterogeneousmultiagenttaskplanningmethodincomplexmarineenvironment |