Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning
In this paper, an enhanced deep reinforcement learning approach is presented for unmanned aerial vehicles (UAVs) operating in dynamic and potentially hazardous environments. Initially, the capability to discern obstacles from visual data is achieved through the application of the Yolov8-StrongSort t...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/13/2/162 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849718709700001792 |
|---|---|
| author | Jiandong Liu Wei Luo Guoqing Zhang Ruihao Li |
| author_facet | Jiandong Liu Wei Luo Guoqing Zhang Ruihao Li |
| author_sort | Jiandong Liu |
| collection | DOAJ |
| description | In this paper, an enhanced deep reinforcement learning approach is presented for unmanned aerial vehicles (UAVs) operating in dynamic and potentially hazardous environments. Initially, the capability to discern obstacles from visual data is achieved through the application of the Yolov8-StrongSort technique. Concurrently, a novel data storage system for deep Q-networks (DQN), named dynamic data memory (DDM), is introduced to hasten the learning process and convergence for UAVs. Furthermore, addressing the issue of UAVs’ paths veering too close to obstacles, a novel strategy employing an artificial potential field to adjust the reward function is introduced, which effectively guides the UAVs away from proximate obstacles. Rigorous simulation tests in an AirSim-based environment confirm the effectiveness of these methods. Compared to DQN, dueling DQN, M-DQN, improved Q-learning, DDM-DQN, EPF (enhanced potential field), APF-DQN, and L1-MBRL, our algorithm achieves the highest success rate of 77.67%, while also having the lowest average number of moving steps. Additionally, we conducted obstacle avoidance experiments with UAVs with different densities of obstacles. These tests highlight fast learning convergence and real-time obstacle detection and avoidance, ensuring successful achievement of the target. |
| format | Article |
| id | doaj-art-3d96d8b91d544f4f99f8244b158cece4 |
| institution | DOAJ |
| issn | 2075-1702 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj-art-3d96d8b91d544f4f99f8244b158cece42025-08-20T03:12:19ZengMDPI AGMachines2075-17022025-02-0113216210.3390/machines13020162Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement LearningJiandong Liu0Wei Luo1Guoqing Zhang2Ruihao Li3North China Institute of Aerospace Engineering, School of Remote Sensing and Information Engineering, Langfang 065000, ChinaNorth China Institute of Aerospace Engineering, School of Remote Sensing and Information Engineering, Langfang 065000, ChinaNorth China Institute of Aerospace Engineering, School of Remote Sensing and Information Engineering, Langfang 065000, ChinaNorth China Institute of Aerospace Engineering, School of Remote Sensing and Information Engineering, Langfang 065000, ChinaIn this paper, an enhanced deep reinforcement learning approach is presented for unmanned aerial vehicles (UAVs) operating in dynamic and potentially hazardous environments. Initially, the capability to discern obstacles from visual data is achieved through the application of the Yolov8-StrongSort technique. Concurrently, a novel data storage system for deep Q-networks (DQN), named dynamic data memory (DDM), is introduced to hasten the learning process and convergence for UAVs. Furthermore, addressing the issue of UAVs’ paths veering too close to obstacles, a novel strategy employing an artificial potential field to adjust the reward function is introduced, which effectively guides the UAVs away from proximate obstacles. Rigorous simulation tests in an AirSim-based environment confirm the effectiveness of these methods. Compared to DQN, dueling DQN, M-DQN, improved Q-learning, DDM-DQN, EPF (enhanced potential field), APF-DQN, and L1-MBRL, our algorithm achieves the highest success rate of 77.67%, while also having the lowest average number of moving steps. Additionally, we conducted obstacle avoidance experiments with UAVs with different densities of obstacles. These tests highlight fast learning convergence and real-time obstacle detection and avoidance, ensuring successful achievement of the target.https://www.mdpi.com/2075-1702/13/2/162UAV obstacle avoidanceartificial potential fielddynamic environmentDQN algorithmYolov8 |
| spellingShingle | Jiandong Liu Wei Luo Guoqing Zhang Ruihao Li Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning Machines UAV obstacle avoidance artificial potential field dynamic environment DQN algorithm Yolov8 |
| title | Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning |
| title_full | Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning |
| title_fullStr | Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning |
| title_full_unstemmed | Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning |
| title_short | Unmanned Aerial Vehicle Path Planning in Complex Dynamic Environments Based on Deep Reinforcement Learning |
| title_sort | unmanned aerial vehicle path planning in complex dynamic environments based on deep reinforcement learning |
| topic | UAV obstacle avoidance artificial potential field dynamic environment DQN algorithm Yolov8 |
| url | https://www.mdpi.com/2075-1702/13/2/162 |
| work_keys_str_mv | AT jiandongliu unmannedaerialvehiclepathplanningincomplexdynamicenvironmentsbasedondeepreinforcementlearning AT weiluo unmannedaerialvehiclepathplanningincomplexdynamicenvironmentsbasedondeepreinforcementlearning AT guoqingzhang unmannedaerialvehiclepathplanningincomplexdynamicenvironmentsbasedondeepreinforcementlearning AT ruihaoli unmannedaerialvehiclepathplanningincomplexdynamicenvironmentsbasedondeepreinforcementlearning |