A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles
A novel artificial intelligence-based approach for the direct yaw control (DYC) of an all-wheel drive (AWD) electric vehicle (EV) is proposed in this paper. To improve adaptability and ability to handle nonlinearities via continuous learning, the proposed algorithm is built upon a twin delayed deep...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11077122/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849303609423953920 |
|---|---|
| author | Reza Jafari Pouria Sarhadi Amin Paykani Shady S. Refaat Pedram Asef |
| author_facet | Reza Jafari Pouria Sarhadi Amin Paykani Shady S. Refaat Pedram Asef |
| author_sort | Reza Jafari |
| collection | DOAJ |
| description | A novel artificial intelligence-based approach for the direct yaw control (DYC) of an all-wheel drive (AWD) electric vehicle (EV) is proposed in this paper. To improve adaptability and ability to handle nonlinearities via continuous learning, the proposed algorithm is built upon a twin delayed deep deterministic policy gradient (TD3) reinforcement learning (RL) algorithm for the optimal torque distribution across four wheels of the vehicle. The proposed model-free torque vectoring algorithm performs based on the interaction of an agent with an environment to learn the optimal policy in a reward-driven manner and obtain the ability to dynamically adapt to varying conditions, such as different roads and vehicle speeds. Unlike conventional control methods that rely on precise system modeling and may struggle to adapt under varying conditions, no model of the vehicle is required in the proposed method. This work proposes a model-free RL-based controller with curriculum learning to train the strategy, where the model learns simpler tasks first, progressively increasing difficulty to enhance stability and convergence. A detailed reward function and well-structured actor-critic networks are devised, and the proposed algorithm is compared with a conventional model-based linear quadratic regulator (LQR) approach. A nonlinear model with 7 degrees of freedom is used to model the dynamic behavior of the vehicle in MATLAB/Simulink, and the results are further verified through the implementation of IPG CarMaker under realistic driving scenarios. The performance of the proposed algorithm is studied across different maneuvers, demonstrating reduced yaw rate error and sideslip angle, resulting in enhanced dynamic stability. |
| format | Article |
| id | doaj-art-b44b9e7d17b4458ab523c8fd0616c0fc |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-b44b9e7d17b4458ab523c8fd0616c0fc2025-08-20T03:56:03ZengIEEEIEEE Access2169-35362025-01-011312715012716910.1109/ACCESS.2025.358793811077122A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric VehiclesReza Jafari0https://orcid.org/0000-0003-4704-0723Pouria Sarhadi1https://orcid.org/0000-0002-6004-676XAmin Paykani2https://orcid.org/0000-0002-1344-5549Shady S. Refaat3https://orcid.org/0000-0001-9392-6141Pedram Asef4https://orcid.org/0000-0003-3264-7303School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, U.K.School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, U.K.School of Engineering and Materials Science, Queen Mary University of London, London, U.K.School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, U.K.Department of Mechanical Engineering, Advanced Propulsion Laboratory, University College London, London, U.K.A novel artificial intelligence-based approach for the direct yaw control (DYC) of an all-wheel drive (AWD) electric vehicle (EV) is proposed in this paper. To improve adaptability and ability to handle nonlinearities via continuous learning, the proposed algorithm is built upon a twin delayed deep deterministic policy gradient (TD3) reinforcement learning (RL) algorithm for the optimal torque distribution across four wheels of the vehicle. The proposed model-free torque vectoring algorithm performs based on the interaction of an agent with an environment to learn the optimal policy in a reward-driven manner and obtain the ability to dynamically adapt to varying conditions, such as different roads and vehicle speeds. Unlike conventional control methods that rely on precise system modeling and may struggle to adapt under varying conditions, no model of the vehicle is required in the proposed method. This work proposes a model-free RL-based controller with curriculum learning to train the strategy, where the model learns simpler tasks first, progressively increasing difficulty to enhance stability and convergence. A detailed reward function and well-structured actor-critic networks are devised, and the proposed algorithm is compared with a conventional model-based linear quadratic regulator (LQR) approach. A nonlinear model with 7 degrees of freedom is used to model the dynamic behavior of the vehicle in MATLAB/Simulink, and the results are further verified through the implementation of IPG CarMaker under realistic driving scenarios. The performance of the proposed algorithm is studied across different maneuvers, demonstrating reduced yaw rate error and sideslip angle, resulting in enhanced dynamic stability.https://ieeexplore.ieee.org/document/11077122/Curriculum learningelectric vehiclereinforcement learningtorque vectoringyaw stabilityvehicle dynamics |
| spellingShingle | Reza Jafari Pouria Sarhadi Amin Paykani Shady S. Refaat Pedram Asef A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles IEEE Access Curriculum learning electric vehicle reinforcement learning torque vectoring yaw stability vehicle dynamics |
| title | A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles |
| title_full | A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles |
| title_fullStr | A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles |
| title_full_unstemmed | A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles |
| title_short | A TD3-Based Reinforcement Learning Algorithm With Curriculum Learning for Adaptive Yaw Control in All-Wheel-Drive Electric Vehicles |
| title_sort | td3 based reinforcement learning algorithm with curriculum learning for adaptive yaw control in all wheel drive electric vehicles |
| topic | Curriculum learning electric vehicle reinforcement learning torque vectoring yaw stability vehicle dynamics |
| url | https://ieeexplore.ieee.org/document/11077122/ |
| work_keys_str_mv | AT rezajafari atd3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT pouriasarhadi atd3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT aminpaykani atd3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT shadysrefaat atd3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT pedramasef atd3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT rezajafari td3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT pouriasarhadi td3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT aminpaykani td3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT shadysrefaat td3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles AT pedramasef td3basedreinforcementlearningalgorithmwithcurriculumlearningforadaptiveyawcontrolinallwheeldriveelectricvehicles |