Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles
Abstract Decision-making of connected and automated vehicles (CAV) includes a sequence of driving maneuvers that improve safety and efficiency, characterized by complex scenarios, strong uncertainty, and high real-time requirements. Deep reinforcement learning (DRL) exhibits excellent capability of...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2025-01-01
|
| Series: | Chinese Journal of Mechanical Engineering |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s10033-024-01158-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559818881466368 |
|---|---|
| author | Fuxing Yao Chao Sun Bing Lu Bo Wang Haiyang Yu |
| author_facet | Fuxing Yao Chao Sun Bing Lu Bo Wang Haiyang Yu |
| author_sort | Fuxing Yao |
| collection | DOAJ |
| description | Abstract Decision-making of connected and automated vehicles (CAV) includes a sequence of driving maneuvers that improve safety and efficiency, characterized by complex scenarios, strong uncertainty, and high real-time requirements. Deep reinforcement learning (DRL) exhibits excellent capability of real-time decision-making and adaptability to complex scenarios, and generalization abilities. However, it is arduous to guarantee complete driving safety and efficiency under the constraints of training samples and costs. This paper proposes a Mixture of Expert method (MoE) based on Soft Actor-Critic (SAC), where the upper-level discriminator dynamically decides whether to activate the lower-level DRL expert or the heuristic expert based on the features of the input state. To further enhance the performance of the DRL expert, a buffer zone is introduced in the reward function, preemptively applying penalties before insecure situations occur. In order to minimize collision and off-road rates, the Intelligent Driver Model (IDM) and Minimizing Overall Braking Induced by Lane changes (MOBIL) strategy are designed by heuristic experts. Finally, tested in typical simulation scenarios, MOE shows a 13.75% improvement in driving efficiency compared with the traditional DRL method with continuous action space. It ensures high safety with zero collision and zero off-road rates while maintaining high adaptability. |
| format | Article |
| id | doaj-art-753cdd07562541e2be1b39ab2cf2f8d7 |
| institution | Kabale University |
| issn | 2192-8258 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Chinese Journal of Mechanical Engineering |
| spelling | doaj-art-753cdd07562541e2be1b39ab2cf2f8d72025-01-05T12:10:35ZengSpringerOpenChinese Journal of Mechanical Engineering2192-82582025-01-0138111410.1186/s10033-024-01158-7Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated VehiclesFuxing Yao0Chao Sun1Bing Lu2Bo Wang3Haiyang Yu4National Engineering Laboratory for Electric Vehicles, Beijing Institute of TechnologyNational Engineering Laboratory for Electric Vehicles, Beijing Institute of TechnologyNational Engineering Laboratory for Electric Vehicles, Beijing Institute of TechnologyShenzhen Automotive Research Institute, Beijing Institute of TechnologySchool of Transportation Science and Engineering, Beihang UniversityAbstract Decision-making of connected and automated vehicles (CAV) includes a sequence of driving maneuvers that improve safety and efficiency, characterized by complex scenarios, strong uncertainty, and high real-time requirements. Deep reinforcement learning (DRL) exhibits excellent capability of real-time decision-making and adaptability to complex scenarios, and generalization abilities. However, it is arduous to guarantee complete driving safety and efficiency under the constraints of training samples and costs. This paper proposes a Mixture of Expert method (MoE) based on Soft Actor-Critic (SAC), where the upper-level discriminator dynamically decides whether to activate the lower-level DRL expert or the heuristic expert based on the features of the input state. To further enhance the performance of the DRL expert, a buffer zone is introduced in the reward function, preemptively applying penalties before insecure situations occur. In order to minimize collision and off-road rates, the Intelligent Driver Model (IDM) and Minimizing Overall Braking Induced by Lane changes (MOBIL) strategy are designed by heuristic experts. Finally, tested in typical simulation scenarios, MOE shows a 13.75% improvement in driving efficiency compared with the traditional DRL method with continuous action space. It ensures high safety with zero collision and zero off-road rates while maintaining high adaptability.https://doi.org/10.1186/s10033-024-01158-7Decision-makingSoft Actor-CriticConnected and automated vehicles |
| spellingShingle | Fuxing Yao Chao Sun Bing Lu Bo Wang Haiyang Yu Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles Chinese Journal of Mechanical Engineering Decision-making Soft Actor-Critic Connected and automated vehicles |
| title | Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles |
| title_full | Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles |
| title_fullStr | Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles |
| title_full_unstemmed | Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles |
| title_short | Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles |
| title_sort | mixture of experts framework based on soft actor critic algorithm for highway decision making of connected and automated vehicles |
| topic | Decision-making Soft Actor-Critic Connected and automated vehicles |
| url | https://doi.org/10.1186/s10033-024-01158-7 |
| work_keys_str_mv | AT fuxingyao mixtureofexpertsframeworkbasedonsoftactorcriticalgorithmforhighwaydecisionmakingofconnectedandautomatedvehicles AT chaosun mixtureofexpertsframeworkbasedonsoftactorcriticalgorithmforhighwaydecisionmakingofconnectedandautomatedvehicles AT binglu mixtureofexpertsframeworkbasedonsoftactorcriticalgorithmforhighwaydecisionmakingofconnectedandautomatedvehicles AT bowang mixtureofexpertsframeworkbasedonsoftactorcriticalgorithmforhighwaydecisionmakingofconnectedandautomatedvehicles AT haiyangyu mixtureofexpertsframeworkbasedonsoftactorcriticalgorithmforhighwaydecisionmakingofconnectedandautomatedvehicles |