Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network
Unmanned helicopters (UH) can evade radar detection by flying at ultralow altitudes, so as to conduct raids on targets. Path planning is one of the key technologies to realize UH’s autonomous completion of raid missions. Since the probability of UH being detected by radar varies with height, how to...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-01-01
|
| Series: | International Journal of Aerospace Engineering |
| Online Access: | http://dx.doi.org/10.1155/2022/1360956 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832565716954382336 |
|---|---|
| author | Jiangyi Yao Xiongwei Li Yang Zhang Jingyu Ji Yanchao Wang Yicen Liu |
| author_facet | Jiangyi Yao Xiongwei Li Yang Zhang Jingyu Ji Yanchao Wang Yicen Liu |
| author_sort | Jiangyi Yao |
| collection | DOAJ |
| description | Unmanned helicopters (UH) can evade radar detection by flying at ultralow altitudes, so as to conduct raids on targets. Path planning is one of the key technologies to realize UH’s autonomous completion of raid missions. Since the probability of UH being detected by radar varies with height, how to accurately identify the radar coverage area to avoid crossing has become a difficult problem in UH path planning. Aiming at this problem, a heuristic deep Q-network (H-DQN) algorithm is proposed. First, as part of the comprehensive reward function, a heuristic reward function is designed. The function can generate dynamic rewards in real time according to the environmental information, so as to guide the UH to move closer to the target and at the same time promote the convergence of the algorithm. Second, in order to smooth the flight path, a smoothing reward function is proposed. This function can evaluate the pros and cons of UH’s actions, so as to prompt UH to choose a smoother path for flight. Finally, the heuristic reward function, the smooth reward function, the collision penalty, and the completion reward are weighted and summed to obtain the heuristic comprehensive reward function. Simulation experiments show that the H-DQN algorithm can help UH to effectively avoid the radar coverage area and successfully complete the raid mission. |
| format | Article |
| id | doaj-art-627bb9a6ccce44988c84235267424f84 |
| institution | Kabale University |
| issn | 1687-5974 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | International Journal of Aerospace Engineering |
| spelling | doaj-art-627bb9a6ccce44988c84235267424f842025-02-03T01:06:46ZengWileyInternational Journal of Aerospace Engineering1687-59742022-01-01202210.1155/2022/1360956Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-NetworkJiangyi Yao0Xiongwei Li1Yang Zhang2Jingyu Ji3Yanchao Wang4Yicen Liu5Equipment Simulation Training CenterEquipment Simulation Training CenterEquipment Simulation Training CenterDepartment of UAV EngineeringEquipment Simulation Training CenterState Key Laboratory of Blind Signal ProcessingUnmanned helicopters (UH) can evade radar detection by flying at ultralow altitudes, so as to conduct raids on targets. Path planning is one of the key technologies to realize UH’s autonomous completion of raid missions. Since the probability of UH being detected by radar varies with height, how to accurately identify the radar coverage area to avoid crossing has become a difficult problem in UH path planning. Aiming at this problem, a heuristic deep Q-network (H-DQN) algorithm is proposed. First, as part of the comprehensive reward function, a heuristic reward function is designed. The function can generate dynamic rewards in real time according to the environmental information, so as to guide the UH to move closer to the target and at the same time promote the convergence of the algorithm. Second, in order to smooth the flight path, a smoothing reward function is proposed. This function can evaluate the pros and cons of UH’s actions, so as to prompt UH to choose a smoother path for flight. Finally, the heuristic reward function, the smooth reward function, the collision penalty, and the completion reward are weighted and summed to obtain the heuristic comprehensive reward function. Simulation experiments show that the H-DQN algorithm can help UH to effectively avoid the radar coverage area and successfully complete the raid mission.http://dx.doi.org/10.1155/2022/1360956 |
| spellingShingle | Jiangyi Yao Xiongwei Li Yang Zhang Jingyu Ji Yanchao Wang Yicen Liu Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network International Journal of Aerospace Engineering |
| title | Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network |
| title_full | Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network |
| title_fullStr | Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network |
| title_full_unstemmed | Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network |
| title_short | Path Planning of Unmanned Helicopter in Complex Environment Based on Heuristic Deep Q-Network |
| title_sort | path planning of unmanned helicopter in complex environment based on heuristic deep q network |
| url | http://dx.doi.org/10.1155/2022/1360956 |
| work_keys_str_mv | AT jiangyiyao pathplanningofunmannedhelicopterincomplexenvironmentbasedonheuristicdeepqnetwork AT xiongweili pathplanningofunmannedhelicopterincomplexenvironmentbasedonheuristicdeepqnetwork AT yangzhang pathplanningofunmannedhelicopterincomplexenvironmentbasedonheuristicdeepqnetwork AT jingyuji pathplanningofunmannedhelicopterincomplexenvironmentbasedonheuristicdeepqnetwork AT yanchaowang pathplanningofunmannedhelicopterincomplexenvironmentbasedonheuristicdeepqnetwork AT yicenliu pathplanningofunmannedhelicopterincomplexenvironmentbasedonheuristicdeepqnetwork |