Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains
Abstract We present a data-driven deep reinforcement learning (DRL) method for the optimization of a hierarchically structured control policy that includes the central pattern generator. This method, which is as a whole referred to as the hierarchical reinforcement learning with the central pattern...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-04-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-94163-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850258535548452864 |
|---|---|
| author | Toshiki Watanabe Akihiro Kubo Kai Tsunoda Tatsuya Matsuba Shintaro Akatsuka Yukihiro Noda Hiroaki Kioka Jin Izawa Shin Ishii Yutaka Nakamura |
| author_facet | Toshiki Watanabe Akihiro Kubo Kai Tsunoda Tatsuya Matsuba Shintaro Akatsuka Yukihiro Noda Hiroaki Kioka Jin Izawa Shin Ishii Yutaka Nakamura |
| author_sort | Toshiki Watanabe |
| collection | DOAJ |
| description | Abstract We present a data-driven deep reinforcement learning (DRL) method for the optimization of a hierarchically structured control policy that includes the central pattern generator. This method, which is as a whole referred to as the hierarchical reinforcement learning with the central pattern generator (HRL-CPG), is then evaluated with the expectation of its applicability in real robot controls. We observed that stable gait motions were gained in a reasonably small number of trials and errors. Thus, it can be deduced that our HRL-CPG can be a candidate DRL method that enables dynamical systems such as real or realistic robots to adapt to a variety of environments within a moderate physical time. |
| format | Article |
| id | doaj-art-e7bcae686747480b8c42dcedca831e33 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-e7bcae686747480b8c42dcedca831e332025-08-20T01:56:06ZengNature PortfolioScientific Reports2045-23222025-04-0115111910.1038/s41598-025-94163-2Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrainsToshiki Watanabe0Akihiro Kubo1Kai Tsunoda2Tatsuya Matsuba3Shintaro Akatsuka4Yukihiro Noda5Hiroaki Kioka6Jin Izawa7Shin Ishii8Yutaka Nakamura9Kyoto UniversityAdvanced Telecommunications Research InstituteKyoto UniversityAISIN CorporationAISIN CorporationAISIN CorporationAISIN CorporationAISIN CorporationKyoto UniversityGuardian Robotics Project, RIKENAbstract We present a data-driven deep reinforcement learning (DRL) method for the optimization of a hierarchically structured control policy that includes the central pattern generator. This method, which is as a whole referred to as the hierarchical reinforcement learning with the central pattern generator (HRL-CPG), is then evaluated with the expectation of its applicability in real robot controls. We observed that stable gait motions were gained in a reasonably small number of trials and errors. Thus, it can be deduced that our HRL-CPG can be a candidate DRL method that enables dynamical systems such as real or realistic robots to adapt to a variety of environments within a moderate physical time.https://doi.org/10.1038/s41598-025-94163-2 |
| spellingShingle | Toshiki Watanabe Akihiro Kubo Kai Tsunoda Tatsuya Matsuba Shintaro Akatsuka Yukihiro Noda Hiroaki Kioka Jin Izawa Shin Ishii Yutaka Nakamura Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains Scientific Reports |
| title | Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains |
| title_full | Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains |
| title_fullStr | Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains |
| title_full_unstemmed | Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains |
| title_short | Hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains |
| title_sort | hierarchical reinforcement learning with central pattern generator for enabling a quadruped robot simulator to walk on a variety of terrains |
| url | https://doi.org/10.1038/s41598-025-94163-2 |
| work_keys_str_mv | AT toshikiwatanabe hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT akihirokubo hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT kaitsunoda hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT tatsuyamatsuba hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT shintaroakatsuka hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT yukihironoda hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT hiroakikioka hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT jinizawa hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT shinishii hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains AT yutakanakamura hierarchicalreinforcementlearningwithcentralpatterngeneratorforenablingaquadrupedrobotsimulatortowalkonavarietyofterrains |