Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains
In the field of hexapod robot control, the application of central pattern generators (CPG) and deep reinforcement learning (DRL) is becoming increasingly common. Compared to traditional control methods that rely on dynamic models, both the CPG and the end-to-end DRL approaches significantly simplify...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-09-01
|
| Series: | Biomimetic Intelligence and Robotics |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667379725000221 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849229248312639488 |
|---|---|
| author | Shichang Huang Zhihan Xiao Minhua Zheng Wen Shi |
| author_facet | Shichang Huang Zhihan Xiao Minhua Zheng Wen Shi |
| author_sort | Shichang Huang |
| collection | DOAJ |
| description | In the field of hexapod robot control, the application of central pattern generators (CPG) and deep reinforcement learning (DRL) is becoming increasingly common. Compared to traditional control methods that rely on dynamic models, both the CPG and the end-to-end DRL approaches significantly simplify the complexity of designing control models. However, relying solely on DRL for control also has its drawbacks, such as slow convergence speed and low exploration efficiency. Moreover, although the CPG can produce rhythmic gaits, its control strategy is relatively singular, limiting the robot’s ability to adapt to complex terrains. To overcome these limitations, this study proposes a three-layer DRL control architecture. The high-level reinforcement learning controller is responsible for learning the parameters of the middle-level CPG and the low-level mapping functions, while the middle and low level controllers coordinate the joint movements within and between legs. By integrating the learning capabilities of DRL with the gait generation characteristics of CPG, this method significantly enhances the stability and adaptability of hexapod robots in complex terrains. Experimental results show that, compared to pure DRL approaches, this method significantly improves learning efficiency and control performance, when dealing with complex terrains, it considerably enhances the robot’s stability and adaptability compared to pure CPG control. |
| format | Article |
| id | doaj-art-8eadcf3ef1a94250b39066b40d7acc4d |
| institution | Kabale University |
| issn | 2667-3797 |
| language | English |
| publishDate | 2025-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Biomimetic Intelligence and Robotics |
| spelling | doaj-art-8eadcf3ef1a94250b39066b40d7acc4d2025-08-22T04:58:39ZengElsevierBiomimetic Intelligence and Robotics2667-37972025-09-015310023110.1016/j.birob.2025.100231Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrainsShichang Huang0Zhihan Xiao1Minhua Zheng2Wen Shi3School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, ChinaSchool of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, ChinaSchool of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, China; The Key Laboratory of Vehicle Advanced Manufacturing, Measuring and Control Technology, Ministry of Education, Beijing Jiaotong University, Beijing, 100044 China; Corresponding author at: School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, China.Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaIn the field of hexapod robot control, the application of central pattern generators (CPG) and deep reinforcement learning (DRL) is becoming increasingly common. Compared to traditional control methods that rely on dynamic models, both the CPG and the end-to-end DRL approaches significantly simplify the complexity of designing control models. However, relying solely on DRL for control also has its drawbacks, such as slow convergence speed and low exploration efficiency. Moreover, although the CPG can produce rhythmic gaits, its control strategy is relatively singular, limiting the robot’s ability to adapt to complex terrains. To overcome these limitations, this study proposes a three-layer DRL control architecture. The high-level reinforcement learning controller is responsible for learning the parameters of the middle-level CPG and the low-level mapping functions, while the middle and low level controllers coordinate the joint movements within and between legs. By integrating the learning capabilities of DRL with the gait generation characteristics of CPG, this method significantly enhances the stability and adaptability of hexapod robots in complex terrains. Experimental results show that, compared to pure DRL approaches, this method significantly improves learning efficiency and control performance, when dealing with complex terrains, it considerably enhances the robot’s stability and adaptability compared to pure CPG control.http://www.sciencedirect.com/science/article/pii/S2667379725000221Hexapod robotCentral pattern generationReinforcement learningComplex terrains |
| spellingShingle | Shichang Huang Zhihan Xiao Minhua Zheng Wen Shi Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains Biomimetic Intelligence and Robotics Hexapod robot Central pattern generation Reinforcement learning Complex terrains |
| title | Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains |
| title_full | Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains |
| title_fullStr | Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains |
| title_full_unstemmed | Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains |
| title_short | Hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains |
| title_sort | hierarchical reinforcement learning for enhancing stability and adaptability of hexapod robots in complex terrains |
| topic | Hexapod robot Central pattern generation Reinforcement learning Complex terrains |
| url | http://www.sciencedirect.com/science/article/pii/S2667379725000221 |
| work_keys_str_mv | AT shichanghuang hierarchicalreinforcementlearningforenhancingstabilityandadaptabilityofhexapodrobotsincomplexterrains AT zhihanxiao hierarchicalreinforcementlearningforenhancingstabilityandadaptabilityofhexapodrobotsincomplexterrains AT minhuazheng hierarchicalreinforcementlearningforenhancingstabilityandadaptabilityofhexapodrobotsincomplexterrains AT wenshi hierarchicalreinforcementlearningforenhancingstabilityandadaptabilityofhexapodrobotsincomplexterrains |