Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method
Mobile robot navigation is a critical aspect of robotics, with applications spanning from service robots to industrial automation. However, navigating in complex and dynamic environments poses many challenges, such as avoiding obstacles, making decisions in real-time, and adapting to new situations....
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/23/11020 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850061272584814592 |
|---|---|
| author | Wen-Chung Cheng Zhen Ni Xiangnan Zhong Minghan Wei |
| author_facet | Wen-Chung Cheng Zhen Ni Xiangnan Zhong Minghan Wei |
| author_sort | Wen-Chung Cheng |
| collection | DOAJ |
| description | Mobile robot navigation is a critical aspect of robotics, with applications spanning from service robots to industrial automation. However, navigating in complex and dynamic environments poses many challenges, such as avoiding obstacles, making decisions in real-time, and adapting to new situations. Reinforcement Learning (RL) has emerged as a promising approach to enable robots to learn navigation policies from their interactions with the environment. However, application of RL methods to real-world tasks such as mobile robot navigation, and evaluating their performance under various training–testing settings has not been sufficiently researched. In this paper, we have designed an evaluation framework that investigates the RL algorithm’s generalization capability in regard to <i>unseen</i> scenarios in terms of learning convergence and success rates by transferring learned policies in simulation to physical environments. To achieve this, we designed a simulated environment in Gazebo for training the robot over a high number of episodes. The training environment closely mimics the typical indoor scenarios that a mobile robot can encounter, replicating real-world challenges. For evaluation, we designed physical environments with and without unforeseen indoor scenarios. This evaluation framework outputs statistical metrics, which we then use to conduct an extensive study on a deep RL method, namely the proximal policy optimization (PPO). The results provide valuable insights into the strengths and limitations of the method for mobile robot navigation. Our experiments demonstrate that the trained model from simulations can be deployed to the previously <i>unseen</i> physical world with a success rate of over <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>88</mn><mo>%</mo></mrow></semantics></math></inline-formula>. The insights gained from our study can assist practitioners and researchers in selecting suitable RL approaches and training–testing settings for their specific robotic navigation tasks. |
| format | Article |
| id | doaj-art-b62bb2207d604e5e963769cda1b8604b |
| institution | DOAJ |
| issn | 2076-3417 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-b62bb2207d604e5e963769cda1b8604b2025-08-20T02:50:17ZengMDPI AGApplied Sciences2076-34172024-11-0114231102010.3390/app142311020Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO MethodWen-Chung Cheng0Zhen Ni1Xiangnan Zhong2Minghan Wei3Department of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USADepartment of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USADepartment of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USADepartment of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USAMobile robot navigation is a critical aspect of robotics, with applications spanning from service robots to industrial automation. However, navigating in complex and dynamic environments poses many challenges, such as avoiding obstacles, making decisions in real-time, and adapting to new situations. Reinforcement Learning (RL) has emerged as a promising approach to enable robots to learn navigation policies from their interactions with the environment. However, application of RL methods to real-world tasks such as mobile robot navigation, and evaluating their performance under various training–testing settings has not been sufficiently researched. In this paper, we have designed an evaluation framework that investigates the RL algorithm’s generalization capability in regard to <i>unseen</i> scenarios in terms of learning convergence and success rates by transferring learned policies in simulation to physical environments. To achieve this, we designed a simulated environment in Gazebo for training the robot over a high number of episodes. The training environment closely mimics the typical indoor scenarios that a mobile robot can encounter, replicating real-world challenges. For evaluation, we designed physical environments with and without unforeseen indoor scenarios. This evaluation framework outputs statistical metrics, which we then use to conduct an extensive study on a deep RL method, namely the proximal policy optimization (PPO). The results provide valuable insights into the strengths and limitations of the method for mobile robot navigation. Our experiments demonstrate that the trained model from simulations can be deployed to the previously <i>unseen</i> physical world with a success rate of over <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>88</mn><mo>%</mo></mrow></semantics></math></inline-formula>. The insights gained from our study can assist practitioners and researchers in selecting suitable RL approaches and training–testing settings for their specific robotic navigation tasks.https://www.mdpi.com/2076-3417/14/23/11020automated learning frameworkplatform developmentautonomous robotdeep reinforcement learningphysical implementationcollision avoidance |
| spellingShingle | Wen-Chung Cheng Zhen Ni Xiangnan Zhong Minghan Wei Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method Applied Sciences automated learning framework platform development autonomous robot deep reinforcement learning physical implementation collision avoidance |
| title | Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method |
| title_full | Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method |
| title_fullStr | Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method |
| title_full_unstemmed | Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method |
| title_short | Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method |
| title_sort | autonomous robot goal seeking and collision avoidance in the physical world an automated learning and evaluation framework based on the ppo method |
| topic | automated learning framework platform development autonomous robot deep reinforcement learning physical implementation collision avoidance |
| url | https://www.mdpi.com/2076-3417/14/23/11020 |
| work_keys_str_mv | AT wenchungcheng autonomousrobotgoalseekingandcollisionavoidanceinthephysicalworldanautomatedlearningandevaluationframeworkbasedontheppomethod AT zhenni autonomousrobotgoalseekingandcollisionavoidanceinthephysicalworldanautomatedlearningandevaluationframeworkbasedontheppomethod AT xiangnanzhong autonomousrobotgoalseekingandcollisionavoidanceinthephysicalworldanautomatedlearningandevaluationframeworkbasedontheppomethod AT minghanwei autonomousrobotgoalseekingandcollisionavoidanceinthephysicalworldanautomatedlearningandevaluationframeworkbasedontheppomethod |