Heterogeneous Multi-Agent Task Planning Method in Complex Marine Environment - Shoumin Wang; Ning Niu; Zhichao Wang; Yaxuan Lv; Jing Zhang (IEEE Access)
To enable collaborative scouting / strike / assessment of underwater time-sensitive targets by heterogeneous multi-agent systems, in this study a heterogeneous multi-agent collaborative decision-making method is proposed based on deep reinforcement learning. The method integrates two core learning frameworks one for heterogeneous multi-agent task allocation and one for single-agent multi-task learning. The task allocation framework combines the proximal policy optimization algorithm with experience replay to train an Actor network, ensuring stable iterative updates of task allocation policies toward high-reward directions. Concurrently, a Critic network is trained using successor features and temporal difference methods, establishing a foundation for transfer learning. A region partitioning mechanism is introduced to construct a foundational knowledge base, enabling the transfer of sub-region knowledge acquired by multi-agents to target regions, thereby addressing complex underwater task allocation scenarios. The single-agent multi-task learning framework employs an experience replay pool with knowledge transfer attributes and policy distillation technology, allowing each agent to assimilate task-specific expertise from heterogeneous peers across diverse mission scenarios. This capability supports multi-task execution, including path planning, emergency obstacle avoidance, and trajectory tracking.