Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning
State-of-the-art theoretical models and new traffic signal control technologies are key guarantees for improving the management and safety performance of transportation systems, and multiagent reinforcement learning (MARL) methods have been widely applied in the field of signal control. Researchers...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-01-01
|
| Series: | Journal of Advanced Transportation |
| Online Access: | http://dx.doi.org/10.1155/atr/3890878 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850179472707289088 |
|---|---|
| author | Jixiang Wang Siqi Chen Jing Wei Boao Wang Haiyang Yu |
| author_facet | Jixiang Wang Siqi Chen Jing Wei Boao Wang Haiyang Yu |
| author_sort | Jixiang Wang |
| collection | DOAJ |
| description | State-of-the-art theoretical models and new traffic signal control technologies are key guarantees for improving the management and safety performance of transportation systems, and multiagent reinforcement learning (MARL) methods have been widely applied in the field of signal control. Researchers in the transportation domain have effectively addressed the issues of poor convergence and suboptimal optimization encountered in RL for multi-intersection signal control scenarios by adopting the centralized training with decentralized execution (CTDE) approach. However, due to the heterogeneity among intersections, simply decomposing the global reward into a sum of intersection-level rewards is unreasonable, posing a challenge in balancing the interests of individual intersections and the entire road network. Additionally, the assumption that all intersections within the system make decisions synchronously is rather strong. Therefore, this paper proposes a distributed traffic model tailored for synchronous decision-making and, based on that, introduces an asynchronous decision-making traffic model according to decoupled intersection control. Simulation experiments show that the asynchronous decision-making method proposed in this paper not only improves the model convergence speed by at least 19% compared to the multiagent deep RL (MADRL) algorithm used for synchronous decision-making, but also improves the model by at least 10.5% in vehicle driving speed, maximum queue length, and average queue length within the decodable range (the traffic density is between 100 vehicles/km and 400 vehicles/km). In the same traffic scenario, the MADRL algorithm used for asynchronous decision-making has improved the average vehicle delay and average queue length by at least 55% compared to traditional arterial green wave control methods and adaptive control methods, and by at least 5% compared to SAC and A2C methods. |
| format | Article |
| id | doaj-art-9e83a9d97df547baa1c95f1d124912ae |
| institution | OA Journals |
| issn | 2042-3195 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Journal of Advanced Transportation |
| spelling | doaj-art-9e83a9d97df547baa1c95f1d124912ae2025-08-20T02:18:29ZengWileyJournal of Advanced Transportation2042-31952025-01-01202510.1155/atr/3890878Multi-Intersection Signal Control Based on Asynchronous Reinforcement LearningJixiang Wang0Siqi Chen1Jing Wei2Boao Wang3Haiyang Yu4School of Transportation Science and EngineeringSchool of Transportation Science and EngineeringSchool of Electrical and Control EngineeringSchool of Transportation Science and EngineeringSchool of Transportation Science and EngineeringState-of-the-art theoretical models and new traffic signal control technologies are key guarantees for improving the management and safety performance of transportation systems, and multiagent reinforcement learning (MARL) methods have been widely applied in the field of signal control. Researchers in the transportation domain have effectively addressed the issues of poor convergence and suboptimal optimization encountered in RL for multi-intersection signal control scenarios by adopting the centralized training with decentralized execution (CTDE) approach. However, due to the heterogeneity among intersections, simply decomposing the global reward into a sum of intersection-level rewards is unreasonable, posing a challenge in balancing the interests of individual intersections and the entire road network. Additionally, the assumption that all intersections within the system make decisions synchronously is rather strong. Therefore, this paper proposes a distributed traffic model tailored for synchronous decision-making and, based on that, introduces an asynchronous decision-making traffic model according to decoupled intersection control. Simulation experiments show that the asynchronous decision-making method proposed in this paper not only improves the model convergence speed by at least 19% compared to the multiagent deep RL (MADRL) algorithm used for synchronous decision-making, but also improves the model by at least 10.5% in vehicle driving speed, maximum queue length, and average queue length within the decodable range (the traffic density is between 100 vehicles/km and 400 vehicles/km). In the same traffic scenario, the MADRL algorithm used for asynchronous decision-making has improved the average vehicle delay and average queue length by at least 55% compared to traditional arterial green wave control methods and adaptive control methods, and by at least 5% compared to SAC and A2C methods.http://dx.doi.org/10.1155/atr/3890878 |
| spellingShingle | Jixiang Wang Siqi Chen Jing Wei Boao Wang Haiyang Yu Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning Journal of Advanced Transportation |
| title | Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning |
| title_full | Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning |
| title_fullStr | Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning |
| title_full_unstemmed | Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning |
| title_short | Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning |
| title_sort | multi intersection signal control based on asynchronous reinforcement learning |
| url | http://dx.doi.org/10.1155/atr/3890878 |
| work_keys_str_mv | AT jixiangwang multiintersectionsignalcontrolbasedonasynchronousreinforcementlearning AT siqichen multiintersectionsignalcontrolbasedonasynchronousreinforcementlearning AT jingwei multiintersectionsignalcontrolbasedonasynchronousreinforcementlearning AT boaowang multiintersectionsignalcontrolbasedonasynchronousreinforcementlearning AT haiyangyu multiintersectionsignalcontrolbasedonasynchronousreinforcementlearning |