A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments
Various approaches have been explored to address the path planning problem for mobile robots. However, it remains a significant challenge, particularly in environments where a multi-tasking mobile robot operates alongside stochastically moving humans. This paper focuses on path planning for a mobile...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/14/7812 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849409119066259456 |
|---|---|
| author | Jian Mi Jianwen Liu Yue Xu Zhongjie Long Jun Wang Wei Xu Tao Ji |
| author_facet | Jian Mi Jianwen Liu Yue Xu Zhongjie Long Jun Wang Wei Xu Tao Ji |
| author_sort | Jian Mi |
| collection | DOAJ |
| description | Various approaches have been explored to address the path planning problem for mobile robots. However, it remains a significant challenge, particularly in environments where a multi-tasking mobile robot operates alongside stochastically moving humans. This paper focuses on path planning for a mobile robot executing multiple pickup and delivery tasks in an environment shared with humans. To plan a safe path and achieve high task success rate, a Reinforcement Learning (RL)-based double layer controller is proposed in which a double-layer learning algorithm is developed. The high-level layer integrates a Finite-State Automaton (FSA) with RL to perform global strategy learning and task-level decision-making. The low-level layer handles local path planning by incorporating a Markov Decision Process (MDP) that accounts for environmental uncertainties. We verify the proposed double layer algorithm under different configurations and evaluate its performance based on several metrics, including task success rate, reward, etc. The proposed method outperforms conventional RL in terms of reward (+63.1%) and task success rate (+113.0%). The simulation results demonstrate the effectiveness of the proposed algorithm in solving path planning problem with stochastic human uncertainties. |
| format | Article |
| id | doaj-art-ae128b82bfaf4aea8e716555819e4e1b |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-ae128b82bfaf4aea8e716555819e4e1b2025-08-20T03:35:36ZengMDPI AGApplied Sciences2076-34172025-07-011514781210.3390/app15147812A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared EnvironmentsJian Mi0Jianwen Liu1Yue Xu2Zhongjie Long3Jun Wang4Wei Xu5Tao Ji6Department of Transport Engineering, College of Architecture Science and Engineering, Yangzhou University, Yangzhou 225127, ChinaDepartment of Transport Engineering, College of Architecture Science and Engineering, Yangzhou University, Yangzhou 225127, ChinaDepartment of Transport Engineering, College of Architecture Science and Engineering, Yangzhou University, Yangzhou 225127, ChinaKey Laboratory of the Ministry of Education for Modern Measurement & Control Technology, School of Electromechanical Engineering, Beijing Information Science & Technology University, Beijing 102206, ChinaCollege of Information Science & Technology, Beijing University of Chemical Technology, Beijing 100029, ChinaSmart Lab of Innovation, Sinotrans Innovation & Technology Co., Ltd., Beijing 100029, ChinaDepartment of Transport Engineering, College of Architecture Science and Engineering, Yangzhou University, Yangzhou 225127, ChinaVarious approaches have been explored to address the path planning problem for mobile robots. However, it remains a significant challenge, particularly in environments where a multi-tasking mobile robot operates alongside stochastically moving humans. This paper focuses on path planning for a mobile robot executing multiple pickup and delivery tasks in an environment shared with humans. To plan a safe path and achieve high task success rate, a Reinforcement Learning (RL)-based double layer controller is proposed in which a double-layer learning algorithm is developed. The high-level layer integrates a Finite-State Automaton (FSA) with RL to perform global strategy learning and task-level decision-making. The low-level layer handles local path planning by incorporating a Markov Decision Process (MDP) that accounts for environmental uncertainties. We verify the proposed double layer algorithm under different configurations and evaluate its performance based on several metrics, including task success rate, reward, etc. The proposed method outperforms conventional RL in terms of reward (+63.1%) and task success rate (+113.0%). The simulation results demonstrate the effectiveness of the proposed algorithm in solving path planning problem with stochastic human uncertainties.https://www.mdpi.com/2076-3417/15/14/7812reinforcement learningdouble-layer controlpath planningmobile robotshuman-shared environmentsMDP |
| spellingShingle | Jian Mi Jianwen Liu Yue Xu Zhongjie Long Jun Wang Wei Xu Tao Ji A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments Applied Sciences reinforcement learning double-layer control path planning mobile robots human-shared environments MDP |
| title | A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments |
| title_full | A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments |
| title_fullStr | A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments |
| title_full_unstemmed | A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments |
| title_short | A Reinforcement Learning-Based Double Layer Controller for Mobile Robot in Human-Shared Environments |
| title_sort | reinforcement learning based double layer controller for mobile robot in human shared environments |
| topic | reinforcement learning double-layer control path planning mobile robots human-shared environments MDP |
| url | https://www.mdpi.com/2076-3417/15/14/7812 |
| work_keys_str_mv | AT jianmi areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT jianwenliu areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT yuexu areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT zhongjielong areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT junwang areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT weixu areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT taoji areinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT jianmi reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT jianwenliu reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT yuexu reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT zhongjielong reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT junwang reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT weixu reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments AT taoji reinforcementlearningbaseddoublelayercontrollerformobilerobotinhumansharedenvironments |