Imitation-Reinforcement Learning Penetration Strategy for Hypersonic Vehicle in Gliding Phase - Lei Xu; Yingzi Guan; Jialun Pu; Changzhu Wei (Aerospace)
To enhance the penetration capability of hypersonic vehicles in the gliding phase, an intelligent maneuvering penetration strategy combining imitation learning and reinforcement learning is proposed. Firstly, a reinforcement learning penetration model for hypersonic vehicles is established based on the Markov Decision Process (MDP), with the design of state, action spaces, and composite reward function based on Zero-Effort Miss (ZEM). Furthermore, to overcome the difficulties in training reinforcement learning models, a truncated horizon method is employed to integrate reinforcement learning with imitation learning at the level of the optimization target. This results in the construction of a Truncated Horizon Imitation Learning Soft Actor–Critic (THIL-SAC) intelligent penetration strategy learning model, enabling a smooth transition from imitation to exploration. Finally, reward shaping and expert policies are introduced to enhance the training process. Simulation results demonstrate that the THIL-SAC strategy achieves faster convergence compared to the standard SAC method and outperforms expert strategies. Additionally, the THIL-SAC strategy meets real-time requirements for high-speed penetration scenarios, offering improved adaptability and penetration performance.