Distributed Reinforcement Learning with GNN Integration for Network-Level Traffic Signal Control

An innovative data-driven AI-based signal control method named “Distributed Artificial intelligence-based Signal Control” (DASIC) has been developed for decentralized training and inference of RL agents on targeted traffic networks. A GNN-based traffic prediction model is developed to represents a learned traffic environment, capture spatiotemporal dynamics and dependencies that propagate along the underlying traffic network, and supplies enriched future traffic state estimates to RL agents. This strategy can effectively address the data scarcity challenge originated from limited offline field observational data availability. In the RL agent, a GNN component is integrated in the RL observation space to allow a RL algorithm to be spatiotemporal-aware. We define the neighborhood of a traffic intersection to be the network view of the intersection, which may be limited by communication network connectivity and reachability. A single controller in a neighborhood has several advantages. First, the RL training can learn from the traffic information flow propagated along the network topology over time for more efficient local control optimization, instead of only acting upon local observations. More importantly, the neighborhood traffic information helps train each control agent to implicitly learn the coordinated control scenarios using graph-based diffusion. Detailed training, testing, and validation of the developed AI models was conducted in Chattanooga simulation network with 44 signalized intersections. Results of the simulation were used to enhance all components of DASIC’s training-deployment-tuning lifecycle to ensure the produced models and data is reliable and robust for real-world implementation.