An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle
Path following is crucial for enhancing the autonomy of unmanned surface vehicles (USVs) in water monitoring missions. This paper presents an offline reinforcement learning (RL) controller for USVs. The controller employs the soft actor–critic algorithm with a diversified Q-ensemble to optimize the...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Journal of Marine Science and Engineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-1312/12/12/2173 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846104155980038144 |
|---|---|
| author | Zexing Zhou Tao Bao Jun Ding Yihong Chen Zhengyi Jiang Bo Zhang |
| author_facet | Zexing Zhou Tao Bao Jun Ding Yihong Chen Zhengyi Jiang Bo Zhang |
| author_sort | Zexing Zhou |
| collection | DOAJ |
| description | Path following is crucial for enhancing the autonomy of unmanned surface vehicles (USVs) in water monitoring missions. This paper presents an offline reinforcement learning (RL) controller for USVs. The controller employs the soft actor–critic algorithm with a diversified Q-ensemble to optimize the steering control policy using a pre-collected dataset of USV path-following trials. A Markov decision process (MDP) tailored for path following is formulated. The proposed offline RL steering controller, trained on static datasets, demonstrates improved sample efficiency and asymptotic performance due to an expanded ensemble of Q-networks. The accuracy and adaptive learning capabilities of the RL controller are validated through simulations and free-running tests. |
| format | Article |
| id | doaj-art-91d32cb7f0a04790af9a2fcfd13b2a22 |
| institution | Kabale University |
| issn | 2077-1312 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Marine Science and Engineering |
| spelling | doaj-art-91d32cb7f0a04790af9a2fcfd13b2a222024-12-27T14:33:09ZengMDPI AGJournal of Marine Science and Engineering2077-13122024-11-011212217310.3390/jmse12122173An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface VehicleZexing Zhou0Tao Bao1Jun Ding2Yihong Chen3Zhengyi Jiang4Bo Zhang5China Ship Scientific Research Center, Wuxi 214082, ChinaChina Ship Scientific Research Center, Wuxi 214082, ChinaChina Ship Scientific Research Center, Wuxi 214082, ChinaChina Ship Scientific Research Center, Wuxi 214082, ChinaChina Ship Scientific Research Center, Wuxi 214082, ChinaChina Ship Scientific Research Center, Wuxi 214082, ChinaPath following is crucial for enhancing the autonomy of unmanned surface vehicles (USVs) in water monitoring missions. This paper presents an offline reinforcement learning (RL) controller for USVs. The controller employs the soft actor–critic algorithm with a diversified Q-ensemble to optimize the steering control policy using a pre-collected dataset of USV path-following trials. A Markov decision process (MDP) tailored for path following is formulated. The proposed offline RL steering controller, trained on static datasets, demonstrates improved sample efficiency and asymptotic performance due to an expanded ensemble of Q-networks. The accuracy and adaptive learning capabilities of the RL controller are validated through simulations and free-running tests.https://www.mdpi.com/2077-1312/12/12/2173soft actor–criticoffline reinforcement learningunmanned surface vehiclespath following control |
| spellingShingle | Zexing Zhou Tao Bao Jun Ding Yihong Chen Zhengyi Jiang Bo Zhang An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle Journal of Marine Science and Engineering soft actor–critic offline reinforcement learning unmanned surface vehicles path following control |
| title | An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle |
| title_full | An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle |
| title_fullStr | An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle |
| title_full_unstemmed | An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle |
| title_short | An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle |
| title_sort | offline reinforcement learning approach for path following of an unmanned surface vehicle |
| topic | soft actor–critic offline reinforcement learning unmanned surface vehicles path following control |
| url | https://www.mdpi.com/2077-1312/12/12/2173 |
| work_keys_str_mv | AT zexingzhou anofflinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT taobao anofflinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT junding anofflinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT yihongchen anofflinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT zhengyijiang anofflinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT bozhang anofflinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT zexingzhou offlinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT taobao offlinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT junding offlinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT yihongchen offlinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT zhengyijiang offlinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle AT bozhang offlinereinforcementlearningapproachforpathfollowingofanunmannedsurfacevehicle |