Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot
Underactuated systems, such as the Cart–Pole and Acrobot, pose significant control challenges due to their inherent nonlinearity and limited actuation. Traditional control methods often struggle to achieve stable and optimal performance in these complex scenarios. This paper presents a novel stable...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-07-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/13/7/601 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849733492849508352 |
|---|---|
| author | Yi-Jen Mon |
| author_facet | Yi-Jen Mon |
| author_sort | Yi-Jen Mon |
| collection | DOAJ |
| description | Underactuated systems, such as the Cart–Pole and Acrobot, pose significant control challenges due to their inherent nonlinearity and limited actuation. Traditional control methods often struggle to achieve stable and optimal performance in these complex scenarios. This paper presents a novel stable reinforcement learning (RL) approach for underactuated systems, integrating advanced exploration–exploitation mechanisms and a refined policy optimization framework to address instability issues in RL-based control. The proposed method is validated through extensive experiments on two benchmark underactuated systems: the Cart–Pole and Acrobot. In the Cart–Pole task, the method achieves long-term balance with high stability, outperforming traditional RL algorithms such as the Proximal Policy Optimization (PPO) in average episode length and robustness to environmental disturbances. For the Acrobot, the approach enables reliable swing-up and near-vertical stabilization but cannot achieve sustained balance control beyond short time intervals due to residual dynamics and control limitations. A key contribution is the development of a hybrid PPO–sliding mode control strategy that enhances learning efficiency and stabilities for underactuated systems. |
| format | Article |
| id | doaj-art-1494851b92604540b3bbc227ca829916 |
| institution | DOAJ |
| issn | 2075-1702 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj-art-1494851b92604540b3bbc227ca8299162025-08-20T03:08:01ZengMDPI AGMachines2075-17022025-07-0113760110.3390/machines13070601Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and AcrobotYi-Jen Mon0Department of Electronic Engineering, Ming-Chuan University, Guei-Shan District, Taoyuan City 333, TaiwanUnderactuated systems, such as the Cart–Pole and Acrobot, pose significant control challenges due to their inherent nonlinearity and limited actuation. Traditional control methods often struggle to achieve stable and optimal performance in these complex scenarios. This paper presents a novel stable reinforcement learning (RL) approach for underactuated systems, integrating advanced exploration–exploitation mechanisms and a refined policy optimization framework to address instability issues in RL-based control. The proposed method is validated through extensive experiments on two benchmark underactuated systems: the Cart–Pole and Acrobot. In the Cart–Pole task, the method achieves long-term balance with high stability, outperforming traditional RL algorithms such as the Proximal Policy Optimization (PPO) in average episode length and robustness to environmental disturbances. For the Acrobot, the approach enables reliable swing-up and near-vertical stabilization but cannot achieve sustained balance control beyond short time intervals due to residual dynamics and control limitations. A key contribution is the development of a hybrid PPO–sliding mode control strategy that enhances learning efficiency and stabilities for underactuated systems.https://www.mdpi.com/2075-1702/13/7/601deep learningreinforcement controlunderactuatedPPOintelligent controlopen AI |
| spellingShingle | Yi-Jen Mon Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot Machines deep learning reinforcement control underactuated PPO intelligent control open AI |
| title | Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot |
| title_full | Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot |
| title_fullStr | Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot |
| title_full_unstemmed | Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot |
| title_short | Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot |
| title_sort | decoupled reinforcement hybrid ppo sliding control for underactuated systems application to cart pole and acrobot |
| topic | deep learning reinforcement control underactuated PPO intelligent control open AI |
| url | https://www.mdpi.com/2075-1702/13/7/601 |
| work_keys_str_mv | AT yijenmon decoupledreinforcementhybridpposlidingcontrolforunderactuatedsystemsapplicationtocartpoleandacrobot |