Autonomous decision-making method of unmanned ship based on improved DDPG algorithm
ObjectivesTo enhance the safety and efficiency of maritime traffic, this paper proposes an autonomous collision avoidance decision-making method for unmanned ships based on an enhanced Deep Deterministic Policy Gradient (DDPG) algorithm. Methods In order to address the issues of low data utilization...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Editorial Office of Chinese Journal of Ship Research
2025-02-01
|
| Series: | Zhongguo Jianchuan Yanjiu |
| Subjects: | |
| Online Access: | http://www.ship-research.com/en/article/doi/10.19693/j.issn.1673-3185.03929 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850240479606603776 |
|---|---|
| author | Wei GUAN Shuhui HAO Zhewen CUI Miaomiao WANG |
| author_facet | Wei GUAN Shuhui HAO Zhewen CUI Miaomiao WANG |
| author_sort | Wei GUAN |
| collection | DOAJ |
| description | ObjectivesTo enhance the safety and efficiency of maritime traffic, this paper proposes an autonomous collision avoidance decision-making method for unmanned ships based on an enhanced Deep Deterministic Policy Gradient (DDPG) algorithm. Methods In order to address the issues of low data utilization and poor convergence in traditional DDPG algorithms, we employ Priority Experience Replay (PER) to dynamically adjust experience priority, reduce sample correlation, and utilize the Long Short-Term Memory (LSTM) network to improve the algorithm convergence. Based on the domain knowledge of ships and adhering to the International Regulations for Preventing Collisions at Sea (COLREGs), a model for determining meeting situations and a novel set of reward functions that consider urgent scenarios when other ships fail to comply with the COLREGs are introduced. Generalization experiments are conducted involving two-ship and multi-ship encounters to validate the effectiveness of the proposed method.Results As the experimental results demonstrate, compared to traditional DDPG algorithms, our improved approach enhances the convergence speed by approximately 28.8%. Conclusions The trained model enables autonomous decision-making and navigation while ensuring compliance with the COLREGs, thereby providing valuable insights for intelligent decision-making in the field of maritime transportation. |
| format | Article |
| id | doaj-art-3d9a645f6f8a4a38873e1d45a6ca5dba |
| institution | OA Journals |
| issn | 1673-3185 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Editorial Office of Chinese Journal of Ship Research |
| record_format | Article |
| series | Zhongguo Jianchuan Yanjiu |
| spelling | doaj-art-3d9a645f6f8a4a38873e1d45a6ca5dba2025-08-20T02:00:51ZengEditorial Office of Chinese Journal of Ship ResearchZhongguo Jianchuan Yanjiu1673-31852025-02-0120117218010.19693/j.issn.1673-3185.03929ZG3929Autonomous decision-making method of unmanned ship based on improved DDPG algorithmWei GUAN0Shuhui HAO1Zhewen CUI2Miaomiao WANG3Navigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaObjectivesTo enhance the safety and efficiency of maritime traffic, this paper proposes an autonomous collision avoidance decision-making method for unmanned ships based on an enhanced Deep Deterministic Policy Gradient (DDPG) algorithm. Methods In order to address the issues of low data utilization and poor convergence in traditional DDPG algorithms, we employ Priority Experience Replay (PER) to dynamically adjust experience priority, reduce sample correlation, and utilize the Long Short-Term Memory (LSTM) network to improve the algorithm convergence. Based on the domain knowledge of ships and adhering to the International Regulations for Preventing Collisions at Sea (COLREGs), a model for determining meeting situations and a novel set of reward functions that consider urgent scenarios when other ships fail to comply with the COLREGs are introduced. Generalization experiments are conducted involving two-ship and multi-ship encounters to validate the effectiveness of the proposed method.Results As the experimental results demonstrate, compared to traditional DDPG algorithms, our improved approach enhances the convergence speed by approximately 28.8%. Conclusions The trained model enables autonomous decision-making and navigation while ensuring compliance with the COLREGs, thereby providing valuable insights for intelligent decision-making in the field of maritime transportation.http://www.ship-research.com/en/article/doi/10.19693/j.issn.1673-3185.03929unmanned vehiclesdeep deterministic policy gradient (ddpg) algorithmautonomous collision avoidance decision-makingpriorit-ized experience replay (per)international regulations forprevent-ing collisions at sea (colregs)collision avoidance |
| spellingShingle | Wei GUAN Shuhui HAO Zhewen CUI Miaomiao WANG Autonomous decision-making method of unmanned ship based on improved DDPG algorithm Zhongguo Jianchuan Yanjiu unmanned vehicles deep deterministic policy gradient (ddpg) algorithm autonomous collision avoidance decision-making priorit-ized experience replay (per) international regulations forprevent-ing collisions at sea (colregs) collision avoidance |
| title | Autonomous decision-making method of unmanned ship based on improved DDPG algorithm |
| title_full | Autonomous decision-making method of unmanned ship based on improved DDPG algorithm |
| title_fullStr | Autonomous decision-making method of unmanned ship based on improved DDPG algorithm |
| title_full_unstemmed | Autonomous decision-making method of unmanned ship based on improved DDPG algorithm |
| title_short | Autonomous decision-making method of unmanned ship based on improved DDPG algorithm |
| title_sort | autonomous decision making method of unmanned ship based on improved ddpg algorithm |
| topic | unmanned vehicles deep deterministic policy gradient (ddpg) algorithm autonomous collision avoidance decision-making priorit-ized experience replay (per) international regulations forprevent-ing collisions at sea (colregs) collision avoidance |
| url | http://www.ship-research.com/en/article/doi/10.19693/j.issn.1673-3185.03929 |
| work_keys_str_mv | AT weiguan autonomousdecisionmakingmethodofunmannedshipbasedonimprovedddpgalgorithm AT shuhuihao autonomousdecisionmakingmethodofunmannedshipbasedonimprovedddpgalgorithm AT zhewencui autonomousdecisionmakingmethodofunmannedshipbasedonimprovedddpgalgorithm AT miaomiaowang autonomousdecisionmakingmethodofunmannedshipbasedonimprovedddpgalgorithm |