Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward
Locomotion control for quadruped robots has been extensively studied through various methodologies, including model-based, learning-based, and bio-inspired approaches. The integration of these methods is increasingly recognized as a powerful strategy for enhancing locomotion performance. Among them,...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11036717/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850169656113889280 |
|---|---|
| author | Ryosei Seto Guanda Li Kyo Kutsuzawa Dai Owaki Mitsuhiro Hayashibe |
| author_facet | Ryosei Seto Guanda Li Kyo Kutsuzawa Dai Owaki Mitsuhiro Hayashibe |
| author_sort | Ryosei Seto |
| collection | DOAJ |
| description | Locomotion control for quadruped robots has been extensively studied through various methodologies, including model-based, learning-based, and bio-inspired approaches. The integration of these methods is increasingly recognized as a powerful strategy for enhancing locomotion performance. Among them, the Central Pattern Generator-Reinforcement Learning (CPG-RL) framework, which combines Central Pattern Generators (CPG) with reinforcement learning (RL), offers key advantages such as accelerated learning, improved Sim-to-Real transfer, and the ability to learn with a simplified reward function. However, its robustness on uneven terrains remains limited, as learning primarily focuses on tuning the parameters within the CPG. Specifically, because the foot placement is constrained to a pre-designed foot trajectory, the system is unable to stabilize its posture on uneven terrain. In biological systems, posture control is regulated by the brainstem and cerebellum, which send corrective signals to the spinal cord to modulate muscle activity and maintain balance in coordination with the CPG network. Inspired by this mechanism, this study introduces a postural reflex network that seamlessly integrates with the CPG network to enhance stable locomotion on uneven terrains. Notably, this improvement is achieved exclusively through piecewise sequential learning with a simplified reward function. The proposed approach expands the applicability of CPG-based locomotion under unstructured environments and contributes to the advancement of bio-inspired control strategies for quadrupedal locomotion. The proposed method progressively improves CPG-based locomotion on uneven terrain by learning the CPG parameters and the foot placement adjustments relative to the pre-designed trajectory in two-stage learning manner. |
| format | Article |
| id | doaj-art-dfdd741fcefa472e85d022ed90213645 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-dfdd741fcefa472e85d022ed902136452025-08-20T02:20:40ZengIEEEIEEE Access2169-35362025-01-011310610310611410.1109/ACCESS.2025.357978611036717Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple RewardRyosei Seto0Guanda Li1https://orcid.org/0000-0002-7711-0895Kyo Kutsuzawa2https://orcid.org/0000-0002-5326-7847Dai Owaki3https://orcid.org/0000-0003-1217-3892Mitsuhiro Hayashibe4https://orcid.org/0000-0001-6179-5706Department of Robotics, Neuro-Robotics Laboratory, Graduate School of Engineering, Tohoku University, Sendai, JapanDepartment of Robotics, Neuro-Robotics Laboratory, Graduate School of Engineering, Tohoku University, Sendai, JapanDepartment of Robotics, Neuro-Robotics Laboratory, Graduate School of Engineering, Tohoku University, Sendai, JapanDepartment of Robotics, Neuro-Robotics Laboratory, Graduate School of Engineering, Tohoku University, Sendai, JapanDepartment of Robotics, Neuro-Robotics Laboratory, Graduate School of Engineering, Tohoku University, Sendai, JapanLocomotion control for quadruped robots has been extensively studied through various methodologies, including model-based, learning-based, and bio-inspired approaches. The integration of these methods is increasingly recognized as a powerful strategy for enhancing locomotion performance. Among them, the Central Pattern Generator-Reinforcement Learning (CPG-RL) framework, which combines Central Pattern Generators (CPG) with reinforcement learning (RL), offers key advantages such as accelerated learning, improved Sim-to-Real transfer, and the ability to learn with a simplified reward function. However, its robustness on uneven terrains remains limited, as learning primarily focuses on tuning the parameters within the CPG. Specifically, because the foot placement is constrained to a pre-designed foot trajectory, the system is unable to stabilize its posture on uneven terrain. In biological systems, posture control is regulated by the brainstem and cerebellum, which send corrective signals to the spinal cord to modulate muscle activity and maintain balance in coordination with the CPG network. Inspired by this mechanism, this study introduces a postural reflex network that seamlessly integrates with the CPG network to enhance stable locomotion on uneven terrains. Notably, this improvement is achieved exclusively through piecewise sequential learning with a simplified reward function. The proposed approach expands the applicability of CPG-based locomotion under unstructured environments and contributes to the advancement of bio-inspired control strategies for quadrupedal locomotion. The proposed method progressively improves CPG-based locomotion on uneven terrain by learning the CPG parameters and the foot placement adjustments relative to the pre-designed trajectory in two-stage learning manner.https://ieeexplore.ieee.org/document/11036717/Central pattern generatordeep reinforcement learningpostural reflexquadrupedal robots |
| spellingShingle | Ryosei Seto Guanda Li Kyo Kutsuzawa Dai Owaki Mitsuhiro Hayashibe Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward IEEE Access Central pattern generator deep reinforcement learning postural reflex quadrupedal robots |
| title | Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward |
| title_full | Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward |
| title_fullStr | Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward |
| title_full_unstemmed | Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward |
| title_short | Two-Stage Learning of CPG and Postural Reflex Toward Quadruped Locomotion on Uneven Terrain With Simple Reward |
| title_sort | two stage learning of cpg and postural reflex toward quadruped locomotion on uneven terrain with simple reward |
| topic | Central pattern generator deep reinforcement learning postural reflex quadrupedal robots |
| url | https://ieeexplore.ieee.org/document/11036717/ |
| work_keys_str_mv | AT ryoseiseto twostagelearningofcpgandposturalreflextowardquadrupedlocomotiononuneventerrainwithsimplereward AT guandali twostagelearningofcpgandposturalreflextowardquadrupedlocomotiononuneventerrainwithsimplereward AT kyokutsuzawa twostagelearningofcpgandposturalreflextowardquadrupedlocomotiononuneventerrainwithsimplereward AT daiowaki twostagelearningofcpgandposturalreflextowardquadrupedlocomotiononuneventerrainwithsimplereward AT mitsuhirohayashibe twostagelearningofcpgandposturalreflextowardquadrupedlocomotiononuneventerrainwithsimplereward |