A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning
Owing to the inherent complexities of underwater environments, coupled with restrictive observational angles, the precise operation of an underwater manipulator during autonomous tasks is a sizable undertaking. To tackle this issue, this paper proposes a method for autonomous control of an underwate...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Office of Control and Information Technology
2023-12-01
|
| Series: | Kongzhi Yu Xinxi Jishu |
| Subjects: | |
| Online Access: | http://ctet.csrzic.com/thesisDetails#10.13889/j.issn.2096-5427.2023.06.007 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849225041512759296 |
|---|---|
| author | LI Xinyang LU Nibin LYU Shiwei LIU Hairui |
| author_facet | LI Xinyang LU Nibin LYU Shiwei LIU Hairui |
| author_sort | LI Xinyang |
| collection | DOAJ |
| description | Owing to the inherent complexities of underwater environments, coupled with restrictive observational angles, the precise operation of an underwater manipulator during autonomous tasks is a sizable undertaking. To tackle this issue, this paper proposes a method for autonomous control of an underwater manipulator, leveraging the robust adaptive capacity of reinforcement learning algorithms. Initially, a reinforced learning approach is developed using proximal policy optimization (PPO) intertwined with an actor-critic (AC) algorithm to formulate the autonomous control strategy. Subsequently, an artificial potential field-based reward shaping method is introduced to address the sparse reward predicament apparent throughout the training process. Lastly, a simulation experiment validates the devised control strategy trained using the aforementioned procedures. The verification results illustrate that the strategy competently converges and commands the autonomous movement of the underwater manipulator towards intended targets. It allows for quick, fluid transitions and provides a smooth, stable track for the end effector. |
| format | Article |
| id | doaj-art-9db684ef312840e48dd65962f4dd1e41 |
| institution | Kabale University |
| issn | 2096-5427 |
| language | zho |
| publishDate | 2023-12-01 |
| publisher | Editorial Office of Control and Information Technology |
| record_format | Article |
| series | Kongzhi Yu Xinxi Jishu |
| spelling | doaj-art-9db684ef312840e48dd65962f4dd1e412025-08-25T06:47:51ZzhoEditorial Office of Control and Information TechnologyKongzhi Yu Xinxi Jishu2096-54272023-12-01455046990227A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement LearningLI XinyangLU NibinLYU ShiweiLIU HairuiOwing to the inherent complexities of underwater environments, coupled with restrictive observational angles, the precise operation of an underwater manipulator during autonomous tasks is a sizable undertaking. To tackle this issue, this paper proposes a method for autonomous control of an underwater manipulator, leveraging the robust adaptive capacity of reinforcement learning algorithms. Initially, a reinforced learning approach is developed using proximal policy optimization (PPO) intertwined with an actor-critic (AC) algorithm to formulate the autonomous control strategy. Subsequently, an artificial potential field-based reward shaping method is introduced to address the sparse reward predicament apparent throughout the training process. Lastly, a simulation experiment validates the devised control strategy trained using the aforementioned procedures. The verification results illustrate that the strategy competently converges and commands the autonomous movement of the underwater manipulator towards intended targets. It allows for quick, fluid transitions and provides a smooth, stable track for the end effector.http://ctet.csrzic.com/thesisDetails#10.13889/j.issn.2096-5427.2023.06.007underwater manipulatorreinforcement learningreward shapingautonomous operation |
| spellingShingle | LI Xinyang LU Nibin LYU Shiwei LIU Hairui A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning Kongzhi Yu Xinxi Jishu underwater manipulator reinforcement learning reward shaping autonomous operation |
| title | A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning |
| title_full | A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning |
| title_fullStr | A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning |
| title_full_unstemmed | A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning |
| title_short | A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning |
| title_sort | study on autonomous control of underwater manipulator autonomous operation based on deep reinforcement learning |
| topic | underwater manipulator reinforcement learning reward shaping autonomous operation |
| url | http://ctet.csrzic.com/thesisDetails#10.13889/j.issn.2096-5427.2023.06.007 |
| work_keys_str_mv | AT lixinyang astudyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT lunibin astudyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT lyushiwei astudyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT liuhairui astudyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT lixinyang studyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT lunibin studyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT lyushiwei studyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning AT liuhairui studyonautonomouscontrolofunderwatermanipulatorautonomousoperationbasedondeepreinforcementlearning |