A Deep Reinforcement Learning–Based Urban Traffic Control Model for Vehicle-to-Everything Ecosystem - Lingyu Zheng; Han Chen; Yajie Zou (Journal of Advanced Transportation)
Infrastructure-to-infrastructure (I2I) communication enables the exchange of traffic data between intersections, which brings a new challenge to urban traffic control. This paper proposes a novel deep reinforcement learning (DRL) framework for urban traffic signal control within the vehicle-to-everything (V2X) ecosystem. The framework incorporates a joint-state representation integrating traffic data from both I2I and vehicle-to-infrastructure (V2I) communications, while considering communication range effects. The reward function is designed to optimize both local intersection conditions and global network performance, facilitating adaptive signal coordination. A simulation case study in Huzhou, China, evaluates the proposed model against conventional pretimed, actuated, nonjoint-state DRL, and joint-state reinforcement learning (RL) models. Results demonstrate superior performance of the joint-state DRL model in episode rewards, average speed, and average time loss, particularly during peak traffic periods. To address data limitations, Monte Carlo cross-validation (MCCV) is employed, further validating the model’s robustness. Results show consistent performance advantages in average speed and time loss across peak and off-peak periods, with slight variations compared to joint-state RL models in certain intervals. The impacts of the communication range are also discussed with the proposed model. Pearson correlation analysis reveals a strong positive correlation between the communication range and convergence time across all traffic periods. Meanwhile, correlations between the communication range and reward, average speed, and average time loss vary by traffic period. Findings highlight the transformative potential of integrating DRL with V2X communication technologies for enhancing traffic signal control in complex urban environments. The proposed model offers a flexible, adaptive approach to traffic management, optimizing flow while maintaining safety standards, with implications for future smart city developments.