ETLight: An Evolution Transformer for Efficient Traffic Signal Control

Traffic signal control (TSC) is still one of the most challenging and promising research issues in the field of transportation. Since traditional methods have difficulty in handling dynamically changing traffic flows, reinforcement learning (RL) methods have been introduced into TSC. However, the cost of practical application is critically high due to multiple sampling trials and long learning process. The Transformer architecture has recently attained remarkable results in natural language processing (NLP), but when applied to the field of RL, the standard Transformer architecture is difficult to optimize and faces the problem of hyperparameter sensitivity. In the paper, we transform TSC into a sequence modeling issue and propose a new evolution Transformer architecture to adjust the autoregressive model through reward, past states and actions in the traffic environment to directly generate the best predicted action. In addition, we use the feature evolution module (FEM) instead of residual connections to make the learning process more stable and efficient. Through experiments on public datasets, we demonstrate that our ETLight model achieves a state-of-the-art (SOTA): 1) It achieves the overall best performance on average travel time (ATT) metric, with improvements of up to 6.85%, 3.73% and 3.10% over the best conventional, RL and Transformer methods, respectively; 2) It has a more stable learning process, faster learning speed and better convergence compared to published TSC methods so far; and; 3) it has good robustness and is less sensitive to hyperparameter selection.