Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning
Unmanned aerial vehicles (UAVs) have recently been applied in several contexts due to their flexibility, mobility, and fast deployment. One of the essential aspects of multi-UAV systems is path planning, which autonomously determines paths for drones from starting points to destination points. Howev...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-08-01
|
| Series: | Journal of Sensor and Actuator Networks |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2224-2708/13/5/50 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850206161268113408 |
|---|---|
| author | Zainab AlMania Tarek Sheltami Gamil Ahmed Ashraf Mahmoud Abdulaziz Barnawi |
| author_facet | Zainab AlMania Tarek Sheltami Gamil Ahmed Ashraf Mahmoud Abdulaziz Barnawi |
| author_sort | Zainab AlMania |
| collection | DOAJ |
| description | Unmanned aerial vehicles (UAVs) have recently been applied in several contexts due to their flexibility, mobility, and fast deployment. One of the essential aspects of multi-UAV systems is path planning, which autonomously determines paths for drones from starting points to destination points. However, UAVs face many obstacles in their routes, potentially causing loss or damage. Several heuristic approaches have been investigated to address collision avoidance. These approaches are generally applied in static environments where the environment is known in advance and paths are generated offline, making them unsuitable for unknown or dynamic environments. Additionally, limited flight times due to battery constraints pose another challenge in multi-UAV path planning. Reinforcement learning (RL) emerges as a promising candidate to generate collision-free paths for drones in dynamic environments due to its adaptability and generalization capabilities. In this study, we propose a framework to provide a novel solution for multi-UAV path planning in a 3D dynamic environment. The improved particle swarm optimization with reinforcement learning (IPSO-RL) framework is designed to tackle the multi-UAV path planning problem in a fully distributed and reactive manner. The framework integrates IPSO with deep RL to provide the drone with additional feedback and guidance to operate more sustainably. This integration incorporates a unique reward system that can adapt to various environments. Simulations demonstrate the effectiveness of the IPSO-RL approach, showing superior results in terms of collision avoidance, path length, and energy efficiency compared to other benchmarks. The results also illustrate that the proposed IPSO-RL framework can acquire a feasible and effective route successfully with minimum energy consumption in complicated environments. |
| format | Article |
| id | doaj-art-952cfdff6f2e41fbaff3a6f93bd12efb |
| institution | OA Journals |
| issn | 2224-2708 |
| language | English |
| publishDate | 2024-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Sensor and Actuator Networks |
| spelling | doaj-art-952cfdff6f2e41fbaff3a6f93bd12efb2025-08-20T02:10:54ZengMDPI AGJournal of Sensor and Actuator Networks2224-27082024-08-011355010.3390/jsan13050050Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement LearningZainab AlMania0Tarek Sheltami1Gamil Ahmed2Ashraf Mahmoud3Abdulaziz Barnawi4Computer Engineering Department, Interdisciplinary Research Center of Smart Mobility and Logistics, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi ArabiaComputer Engineering Department, Interdisciplinary Research Center of Smart Mobility and Logistics, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi ArabiaComputer Engineering Department, Interdisciplinary Research Center of Smart Mobility and Logistics, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi ArabiaComputer Engineering Department, Interdisciplinary Research Center of Smart Mobility and Logistics, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi ArabiaComputer Engineering Department, Interdisciplinary Research Center for Intelligent Secure Systems, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi ArabiaUnmanned aerial vehicles (UAVs) have recently been applied in several contexts due to their flexibility, mobility, and fast deployment. One of the essential aspects of multi-UAV systems is path planning, which autonomously determines paths for drones from starting points to destination points. However, UAVs face many obstacles in their routes, potentially causing loss or damage. Several heuristic approaches have been investigated to address collision avoidance. These approaches are generally applied in static environments where the environment is known in advance and paths are generated offline, making them unsuitable for unknown or dynamic environments. Additionally, limited flight times due to battery constraints pose another challenge in multi-UAV path planning. Reinforcement learning (RL) emerges as a promising candidate to generate collision-free paths for drones in dynamic environments due to its adaptability and generalization capabilities. In this study, we propose a framework to provide a novel solution for multi-UAV path planning in a 3D dynamic environment. The improved particle swarm optimization with reinforcement learning (IPSO-RL) framework is designed to tackle the multi-UAV path planning problem in a fully distributed and reactive manner. The framework integrates IPSO with deep RL to provide the drone with additional feedback and guidance to operate more sustainably. This integration incorporates a unique reward system that can adapt to various environments. Simulations demonstrate the effectiveness of the IPSO-RL approach, showing superior results in terms of collision avoidance, path length, and energy efficiency compared to other benchmarks. The results also illustrate that the proposed IPSO-RL framework can acquire a feasible and effective route successfully with minimum energy consumption in complicated environments.https://www.mdpi.com/2224-2708/13/5/50IoDsIPSOreinforcement learningQ-learningactor–critic |
| spellingShingle | Zainab AlMania Tarek Sheltami Gamil Ahmed Ashraf Mahmoud Abdulaziz Barnawi Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning Journal of Sensor and Actuator Networks IoDs IPSO reinforcement learning Q-learning actor–critic |
| title | Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning |
| title_full | Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning |
| title_fullStr | Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning |
| title_full_unstemmed | Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning |
| title_short | Energy-Efficient Online Path Planning for Internet of Drones Using Reinforcement Learning |
| title_sort | energy efficient online path planning for internet of drones using reinforcement learning |
| topic | IoDs IPSO reinforcement learning Q-learning actor–critic |
| url | https://www.mdpi.com/2224-2708/13/5/50 |
| work_keys_str_mv | AT zainabalmania energyefficientonlinepathplanningforinternetofdronesusingreinforcementlearning AT tareksheltami energyefficientonlinepathplanningforinternetofdronesusingreinforcementlearning AT gamilahmed energyefficientonlinepathplanningforinternetofdronesusingreinforcementlearning AT ashrafmahmoud energyefficientonlinepathplanningforinternetofdronesusingreinforcementlearning AT abdulazizbarnawi energyefficientonlinepathplanningforinternetofdronesusingreinforcementlearning |