2.1. Basics of Vehicular Communication

CV communication is an emerging technological framework that aims at direct data transmission between vehicles to vehicles (V2V) and vehicles to road infrastructure (V2I) using wireless technology. It is generally known that CV communication standards consist of three components: (1) IEEE 1609 “Family of Trial-Use Standards for Wireless Access in Vehicular Environments (WAVE)” [27], (2) IEEE 802.11p “Standard for Information Technology” [28], and (3) Society of Automotive Engineers International (SAE) J2735 “DSRC Message Set Dictionary” [29]. These standards are sometimes called WAVE/DSRC for short. The IEEE 1609 family defines an overall structure of the WAVE interface. IEEE 802.11p deals with the multichannel operations of the MAC layer. SAE J2735 defines the framework of DSRC messages, for example, here-I-am (HIA), a-la-carte (ALC), and signal phase and timing (SPaT) messages, to ensure interoperability among any possible CV applications. For safety applications, SAE J2735 also defines the Basic Safety Message (BSM), which is the most fundamental building block that enables proximity awareness. Such framework includes the elements and the usages of possible messages categorized based on the types of applications.

The unique difference of WAVE/DSRC as compared to existing wireless communication standards is that it does not require an authentication procedure. That is, under existing wireless LAN protocols, like IEEE 802.11a/b/g, a mobile node (i.e., a laptop or a smart phone) must be identified by an access point (AP) to join the network. However, these identification steps normally take a few seconds or even minutes and are thus not suitable for a mobile network composed of fast-moving vehicles. Thus, by omitting such identification steps, WAVE/DSRC enables quick connections between transceivers.

The channel operation of WAVE/DSRC is also of interest. The DSRC has a 75 MHz frequency spectrum at 5.9 GHz frequency band. The spectrum is divided every 10 MHz, resulting in seven different channels from 172 to 184 [28]. Channel 178 is called a control channel (CCH), and the other channels are called service channels (SCHs), except for the two channels at both ends that are reserved for future use. While the CCH is dedicated to the transmission of control messages such as beaconing or urgent safety-related messages, the SCHs are designed for exchanging any data packets including vehicular mobility information or commercial services. Therefore, the control messages or safety-related data that are transmitted through the CCH are not affected by data transmission through a SCH, thereby enabling the system to be suitable for the fast-moving mobile network.

On the basis of the above technologies, each vehicle transmits its temporary ID, location, speed, heading, lateral and longitudinal acceleration, brake system status, and vehicle size to surrounding vehicles and the infrastructure. By “listening” to these messages, a signal controller could gain a more comprehensive understanding of the movements of nearby vehicles than with traditional point detectors (e.g., loop detectors). Substantial improvement was seen in several real-world pilot projects such as Audi [30] and BMW [31].

2.2. Data Requirement of Traffic Signal Control

Traffic signal optimization algorithms need appropriate input including both static information about road facilities and real-time dynamic traffic states. Static information is usually preset in optimization models, while dynamic information depends on real-time communication between vehicles to vehicles and vehicles to infrastructures. In the reviewed literatures, the used real-time traffic information can be categorized into aggregative and individual vehicle information. The advantage of aggregative information lies in modest resource requirements such as average transmission content and computation power. In contrast, the upside of individual information is fine-grained considerations of the optimization problem. Examples of aggregative information include total vehicle counts, equipped vehicle ratio [32], and queue length in the lanes related to each traffic lights phase [33, 34]. Required individual vehicle information includes real-time location [35], speed [36], heading [37], and acceleration [38] as well as vehicle type [39] of each vehicle within a certain spatial range. Sometimes, weather and minor events [40] are also taken into account to improve the operations.

2.3. Control Objectives

In the traffic signal timing optimization process, one or more of the measures of effectiveness (MOEs) are optimized under certain constraints to determine appropriate signal timing parameters including phase plans, cycle lengths, green splits, and offsets. Different objective functions and their combinations are used to define the problem. Therefore, proper selection of the objective function for signal timing optimization is a very important task. Minimizing time lost in the intersections, such as average delay, average waiting time, or travel time, is the most commonly selected objective function [41–43]. Some of the studies also considered the variance of waiting time to improve fairness [44]. Several studies conducted queue-based optimization, that is, minimizing average or maximum queue length [34, 45]. In order to consider the environmental influence, decreasing pollutant emissions can be imported as an objective function [46]. It is noted that the mentioned objectives are not independent but interrelated with each other. For example, it is discovered that minimizing total delay can also minimize the total number of stops and, hence, is more suitable for implementation [47]. To consider multiple factors at the same time, special performance index (PI), which is usually defined as the weighted sum of several variables, can also be used in optimization model. For instance, Goodall et al. optimized signal timing plans by minimizing a combination of delay, stops, and decelerations [37]. There are also studies with the objective function considering the minimization of fuel consumption such as [48].

In the reviewed literatures, an unsolved problem is the selection of objective function under different application scenarios. The problem has been stated in a recent research [49].

3.1. Rule-Based Models

Rule-based models are defined as the models which determine the control parameters through specific equations built based on certain optimization criteria. This type of models mostly inherited traditional fix-time control algorithms, replacing the historical traffic data with real-time traffic states awareness under the CV environment.

More extensions reflect in estimating the traffic states in the near future and developing network-wide control. Lee et al. [32] proposed a cumulative travel-time responsive (CTR) real-time intersection control algorithm. The core of the algorithm is based on a stochastic state estimation technique utilizing Kalman filtering that is used in estimating the cumulative travel times under imperfect market penetration rates at every update interval. Compared to an optimized actuated control algorithm, the proposed CTR algorithm improved the total delay time and average speed of the intersection by 34 and 36%, respectively, at 100% market penetration. Bani Younes et al. [53] designed an ITL scheduling algorithm (ITLC) which utilizes vehicular ad hoc communications technology to gather the real-time traffic characteristics of all competing flows of traffic at each signalized road intersection. Further, the ATL algorithm was introduced for open-network control scenarios aiming at high traffic fluency for the arterial flows. Evaluations revealed that the ATL algorithm decreases the average queuing delay at each traffic light by 10% compared with previously introduced traffic scheduling systems. Lin et al. [54] proposed an algorithm that limits the boundary flow of a road network based on MFD and controls the maximum queuing length of each boundary section to avoid the overflow phenomenon.

Rule-based models have the advantages of conciseness and computational convenience. However, the coarse-grained modeling usually contains too much simplification, which compromises the accuracy of the formulation.

3.2. Mathematical Programming-Based Models

Mathematical programming, which means the selection of a best element with regard to some criterion from the feasible region [55], is the most commonly used method to optimize the timing plan of intersection signal control. As a classic paradigm in the field of transportation research, considerable studies have made great contributions in this regard. Some representative studies are reviewed below.

Zhu and Ukkusuri [56] developed a linear programming formulation for autonomous intersection control (LPAIC) accounting for traffic dynamics within a CV environment. Firstly, a lane-based bilevel optimization model is introduced to propagate traffic flows in the network, accounting for dynamic departure time, dynamic route choice, and autonomous intersection control in the context of system optimum network model. Then the bilevel optimization model is transformed to the linear programming formulation by relaxing the nonlinear constraints with a set of linear inequalities. Also, the LPAIC formulation propagates traffic flow consistently as LTM and CTM. The control system is tested in both isolated and grid network scenarios. The simulation result shows that the proposed LPAIC algorithm produces lower total travel time in different V/C than an actuated longest queue first signal control algorithm.

Feng et al. [57] presented a real-time adaptive signal phase allocation algorithm using connected-vehicle data. The proposed algorithm optimizes the phase sequence and duration by solving a two-level optimization problem to minimize total vehicle delay and queue length. A real-world intersection is modeled in VISSIM to validate the algorithms. The results show that the proposed control algorithm outperforms actuated control by reducing total delay by as much as 16.33% in a high penetration rate case.

Li et al. [48] develop a modeling framework for optimizing the timing of a set of traffic signals by considering individual vehicle characteristics such as fuel consumption and travel time. The proposed strategy applies the intelligent driving model (IDM) to predict vehicle trajectories under the connected-vehicle environment. The resulting model is a mixed-integer nonlinear program. Mixed-integer linear or nonlinear model is also frequently adopted in numerous studies, especially studies referring to priority control [39, 58–60].

Priemer and Friedrich [61] presented a traffic control strategy which is phase-based as traditional traffic signal control methods but operates without common parameters like cycle times, offsets, or other fixed timings. In each discrete interval of 5 s, the control algorithm forecasts the future queue length for the next 20 seconds by listening to the received vehicles position and speed data. Within each of these optimization horizons, the method determines the optimal phase sequence in order to reduce the total queue length at an intersection by using the methods of dynamic programming (DP) and complete enumeration (CE) algorithm. Approximate dynamic programming, a variant DP method which allows the controller to learn from its own performance progressively, is adopted in a study by Cai et al. [62]. This study presents a method, VICAC, which combines travel-time estimation and adaptive traffic signal control. Li and Ban [63] decomposed the signal optimization and coordination problem into two levels: an intersection level to optimize phase durations using dynamic programming (DP) and a corridor level to optimize the offsets of all intersections.

Pandit et al. [36] formulate the vehicular traffic signal control problem as a job scheduling problem on processors, with jobs corresponding to platoons of vehicles. Then the oldest job first (OJF) algorithm is used to minimize the delay across the intersection. The evaluation result shows that the algorithm reduces the delays experienced by vehicles as they pass through the intersection under light and medium traffic loads as compared with vehicle-actuated methods, Webster′s method, and pretimed signal control methods.

For mathematical programming, the greatest challenge is the nonlinearities of objective functions and constraints. When the objective functions or constraints are complex, we may turn to heuristic algorithms or dynamic programming method, yet the computational burden is a potential problem for real-time control.

3.3. Artificial Intelligence-Based Models

Artificial intelligence (AI) is defined as the study of intelligent agents which perceives its environment and takes actions that maximize its chance of success at some goal. Traffic signal control based on AI is considered as a promising research area, because an AI system does not assume any prior knowledge of a model or the parameters of its dynamics and thus does not need to rely on the expensive and time-consuming model calibration procedures like existing optimization model. Furthermore, the AI system is capable of “learning” from the environment and thus is expected to improve the operations over time and adapt to changes of the environment. This has been proved by some leading researches in recent years [64, 65]. This type of models is very applicable to the traffic signal control under CV environment for the reason that the traffic information can be perceived in real time. Examples of AI approaches used in reviewed studies include multiagent system, fuzzy logic, neural network, and reinforcement learning.

Both vehicles and traffic signal controllers can be deemed as “agent,” named as vehicle agent and control agent. In this perspective, the whole signal control system becomes a so-called multiagent system. The multiagent system used in a lot of studies has advantages including being model-free and coordinated, colearning, and being suitable for parallel processing. Khamis and Gomaa [66] developed an adaptive multiobjective reinforcement learning system for traffic signal control based on a cooperative multiagent framework. In addition, the authors show that using the Bayesian probability interpretation to estimate the parameters of the MDP probabilities can result in a good response to the traffic nonstationarity. Lee and Park [38] proposed a cooperative vehicle intersection control (CVIC) algorithm based on multiagent model that does not require a traffic signal. By eliminating the potential overlaps of vehicular trajectories coming from all conflicting approaches at the intersection, the CVIC algorithm seeks a safe maneuver for each vehicle approaching intersection and manipulates each of them. Also, an additional algorithm was designed to deal with the system failure cases resulting from inevitable trajectory overlaps at the intersection and infeasible solutions. A simulation-based case study implemented on a hypothetical four-way single-lane approach intersection under varying congestion conditions showed that the CVIC algorithm significantly improved intersection performance compared with conventional actuated intersection control: 99% and 33% of stop delay and total travel time reductions, respectively, were achieved. Kari et al. [23] developed an agent-based online adaptive signal control (ASC) strategy. The system demonstrated savings of 5–14% in reducing travel time and 0–5% in reducing system-wide fuel economy in a scenario with constant demand profiles compared with the QEM/HCM-based strategy. Other representative studies include [67, 68].

Reinforcement learning (RL) is a machine learning paradigm, by which a controller’s policy can be optimized through trial-and-error interactions with an environment. The agent-environment interaction can be described as follows. At every time step t, the agent obtains the state of environment s_t. Given s_t, the agent will decide the next action a_t. Then the environment transits to a new state, s_t+1, for which a reward r_t+1 is given to the agent. The goal of reinforcement learning problem is to find the optimal policy which yields the highest total reward. A schematic illustration is shown in Figure 1. Reinforcement learning is particularly well suited to problems which include a long-term versus short-term reward trade-off. It has been applied successfully to various problems, including robot control, elevator scheduling, telecommunications, backgammon, checkers, and go (AlphaGo).

Liu et al. [69] presented distributed cooperative reinforcement learning-based traffic control that integrates V2X networks’ dynamic clustering algorithm. A dynamic clustering algorithm is proposed based on the enhanced affinity propagation. By integrating the clustering algorithm, a cooperative reinforcement learning control scheme is proposed to balance the traffic load. To address the tough dimensionality curse of reinforcement learning, a distributed mechanism for intersection cooperation is introduced, and a fast gradient-descent function approximation method is proposed to improve the controls’ real-time performance. The proposed algorithm gets the minimum intersection waiting time, serves the most road users, and produces the minimum queue length compared to the baselines. Cheng et al. [44] applied reinforcement learning to fine-tune the control parameters of the network and make it adaptive to various traffic conditions. The algorithm has good performance, especially in high dynamic traffic flows. Yang and Tan [70] provided some results on how the reinforcement learning method performs using Q-matrix and Q-network.

Wang et al. [71] formulated the joint traffic signal and connected vehicle control problem as a reinforcement learning (RL) problem, the action and state spaces of which are specifically designed to take into account the connected vehicles. An effective rewarding mechanism is designed, which takes into account the impact of the detouring on the network traffic efficiency. By utilizing tools from deep RL, an efficient algorithm is proposed to jointly control the traffic signals and the connected vehicles. Numerical results demonstrate validity and efficiency of the models.

Another commonly used intelligent-control model is fuzzy logic—a mathematical system that analyzes analog input values in terms of logical variables that take on continuous values between 0 and 1, in contrast to classical or digital logic, which operates on discrete values of either 1 or 0 (true or false, respectively). Collotta et al. [33] proposed a multicontroller system consisting of a Wireless Sensor Network (WSN), a Phase Sorting Module, and a fuzzy logic controller. The WSN is responsible for the collection of traffic data; the Phase Sorting Module determines the phase execution sequence; each fuzzy controller determines the green time duration for the relevant phase considering the number of enqueued cars in the lanes that are under its control. The fuzzy control strategy is also adopted in [44]. The authors determined the appropriate groups based on real-time traffic conditions using neurofuzzy network.

Before AI-based signal control system is widely implemented, it still faces significant hurdles, despite a large number of studies. A key problem is the lack of interpretability for AI algorithms, especially machine learning algorithms. In addition, some of AI algorithms require tremendous computing power.

3.4. Other Models

Ahmane et al. [45] proposed a model based on Timed Petri Nets with Multipliers (TPNM) which can make the control policy through the structural analysis. The control aims to smooth the traffic through the sequence of vehicles authorized to traverse the intersection. The proposed control policy is based on the modeling of an isolated 4-way intersection as a discrete event dynamic system. The dynamic behavior of the modeled traffic is discrete and represented by a Timed Petri Net with Multipliers (TPNM) in which multipliers are associated with the arcs of the Petri Net. Cheng et al. [72] used the game-theoretic paradigm of fictitious play to iteratively search for a coordinated signal timing plan to be employed, which improves a system-wide performance criterion for a traffic network. Elhenawy et al. [5] proposed a game-theory-based algorithm for controlling autonomous vehicle movements at uncontrolled intersections.

4.1. Different Stages of CV Technologies

According to whether or not autonomous driving technology is equipped, connected vehicles can be divided into manned and automated types. The two types need to be controlled in different ways: manned connected vehicles might be still controlled using traffic signals just as conventional vehicles that cannot communicate with the central controller or other vehicles, while autonomous vehicles can be controlled by coordinated route planning without any traffic signal. In most of the reviewed studies such as [22, 36, 57], the connected vehicles only refer to the manned connected vehicles. There are also studies that aim to solve the route planning problem in a fully autonomous vehicle environment [38, 44]. The situation of two types of vehicles together with conventional vehicles has not been fully studied. One example is the study of [73], in which three categories of vehicles are considered: (1) conventional vehicles, (2) automated vehicles, and (3) manned connected vehicles. The study integrated three different stages of technology development and developed heuristics to switch the signal controls depending on the stage of technology. The simulation results show an evident decrease in the total number of stops and delay when using the connected-vehicle algorithm for the tested scenarios with information level as low as 50%.

Rafter et al. [74] proposed a novel traffic signal control algorithm called Multimode Adaptive Traffic Signals (MATS) which can offer reductions in mean delay for networks with 0–100% connected-vehicle presence. The MATS algorithm combines position information from connected vehicles with data obtained from existing inductive loops and signal timing plans in the network to perform decentralized traffic signal control at urban intersections. The MATS algorithm is capable of adapting to scenarios with low numbers of connected vehicles, an area where existing traffic signal control strategies for connected environments are limited.

4.2. Centralized Control versus Decentralized Control

Majority of signal control systems use centralized formulation and architecture. In such systems, vehicles approaching the intersection communicate with a central controller at the intersection. The central controller optimizes various signal timing parameters of the system at the same time in one mathematical program. However, network signal timing optimization is known as an NP-complete problem and a central optimization technique will not be scalable and applicable to large transportation networks.

The other category of methods uses decentralized control (also called distributed control). Compared with centralized system, the paradigm decomposes the signal timing optimization problem to several interconnected subproblems in different control nodes. Each node only needs to execute a very simple computation, which reduces the complexity of each individual node. For example, Nafi et al. [75] presented a VANET-based road traffic signaling system developed using a distributed architecture by incorporating the distributed networking feature. Ishlam et al. [76] presented a Distributed-Coordinated methodology for signal timing optimization in connected urban street networks, with underlying assumption that all vehicles and intersections are connected, and intersections can share information with each other. The novelty of the work is the decentralized approach, where a mathematical program controls the timing of only a single intersection, which means the approach is in real time and scalable. The results show that the algorithm can increase intersection throughput between 1% and 5% and reduce travel time between 17% and 48%, compared to actuated coordinated signals. Decentralized architecture is also adopted in [5, 37, 51]. But the distributed system has also disadvantages: using distributed cooperative control completely could cause delays in realizing the traffic balance among districts. Therefore, a potential direction is introducing a hierarchical structure to incorporate the centralized control and the distributed cooperative control.

4.3. Isolated Control versus Coordinated Control

There are two distinct modes of traffic signal controller operation: isolated and coordinated. Many studies focus on isolated control because this mode is the foundation of intersection signal control and a significant part of coordinated mode. Liang et al. [77] developed a flexible, real-time traffic signal control algorithm to optimize both phase durations and phase sequences at four-approach intersections with conflicting left turns, based on information obtained from connected vehicles. The location of all connected vehicles is used to identify the presence of nonconnected vehicles that are stopped at the intersection and then identify naturally occurring platoons in the traffic stream. The signal control algorithm then selects the optimal sequence that these platoons should discharge through the intersection to minimize average delay of all identified vehicles. Several heuristic methods are proposed to determine optimal platoon departure sequences in this scenario. Other examples include [22, 37, 47].

The goal of coordinated control is to achieve the global optimum in an arterial or a network, through considering the coordination of all the controllers. Wang et al. [78] developed a joint control model which optimizes the speeds of the connected vehicles and coordinating signals along an arterial simultaneously. This control model forms connected vehicles into platoons so that the vehicles can pass through intersections together with no stops or the least stop time. At the same time, it optimizes signal timing plans along an arterial to achieve lower signal delay and higher throughput. A real-world road network simulation shows that the joint control model can reduce the stop time and stops of coordinate phase by up to 53.69% and 41.15%. The signalized intersection delay per vehicle is reduced by 13.19%. Other representative studies include [39, 68].

4.4. Analysis of Scenarios of CV Deployment with Different Penetration Rates

The quality of traffic signal control under CV environment depends mainly on the number of vehicles equipped with communication devices with respect to the total number of vehicles, the so-called penetration rate. Therefore, various penetration rates should be modeled to obtain the impact factor exactly. He et al. [39] evaluated the performance of the proposed PAMSCOD system and concluded that the system can outperform state-of-practice signal control methods at about a 40% penetration rate. Feng et al. [57] presented a real-time adaptive signal phase allocation algorithm using connected-vehicle data. Due to the low penetration rate of the connected vehicles, an algorithm, EVLS, which estimates the states of unequipped vehicle based on connected-vehicle data is developed to construct a complete arrival table for the phase allocation algorithm. Evaluation shows that the proposed control algorithm outperforms actuated control by reducing total delay by as much as 16.33% in a high penetration rate case and similar delay in a low penetration rate case. Guler et al. [47] found that increase in the penetration rate from 0% up to 60% can significantly reduce the average delay for the proposed algorithm. The above-mentioned studies reveal that the existing control models can outperform the traditional control methods only in a relatively high penetration rate of connected vehicles, which may not be feasible in the near future. How to utilize real-world CV data under low penetration rate environment to improve traffic signal operation is a pressing issue. One of the related studies is that of Zheng and Liu [79]. The authors modeled vehicle arrivals at signalized intersections as a time-dependent Poisson process. An expectation maximization (EM) procedure is derived to solve the parameter estimation problem.

4.5. Priority Control

Priority control aims to improve service and reduce delay for certain traffic mode at intersections. The most common form is transit signal priority (TSP) control. Hu et al. [59] proposed a person-delay-based optimization method for an intelligent TSP logic TSPCV-C that enables bus/signal cooperation and coordination among consecutive signals under the CV environment. The method is evaluated through a computer simulation as well as in the field [60]. A similar form is freight signal priority (FSP). Rather than considering simply travel time and reliability, it is possible to consider vehicle weight, road grade, and truck engine type in order to minimize energy and emissions along a freight corridor [23].

The concept can be generalized for a system in which different traffic modes have different priority level. He et al. [39] presented a unified platoon-based formulation called PAMSCOD to concurrently optimize network traffic signal control for different travel modes given the assumption that advanced communication systems are available between vehicles and traffic controllers. Two modes of traffic composition (transit buses and passenger vehicles) are considered in a decision framework. Microscopic simulation shows that the proposed algorithm can successfully coordinate traffic signals considering the two traffic modes including buses and automobiles and significantly reduce vehicle delay for both modes. The algorithm was improved by the authors in another paper [58]. Liang et al. [80] proposed an algorithm that leverages information from connected vehicles (CVs) arriving at an intersection to identify naturally occurring platoons that consist of both CVs and non-CVs. Simulation tests reveal that the proposed platoon-based algorithm provides superior computational savings (over 95%) compared with algorithms that focus on individual vehicles.

4.6. Performance Evaluation

There are four major elements in the performance evaluation for traffic control optimization: (1) measure of effectiveness (MOE), (2) evaluation platform, (3) evaluation scenarios, and (4) baselines. The widely used MOEs include average delay, stops, queue length, and energy consumption. Most studies use simulation software as evaluation platform such as VISSIM, SUMO, GLD, AIMSUN NG, and Commuter for microscopic traffic simulation and ns2, ns3, NCTUns, and OPNET for network communication simulation. Simulation experiments with different scenarios are carried out by covering varying volume-to-capacity ratios and market penetration rates in most studies. The baselines are usually classic algorithms such as Webster’s method, TRANSYT, HCM method, and Synchro’s optimization method.

It is noted that the car-following model in a connected-vehicle environment may be different from traditional models so appropriate adjustments of the default settings in traffic simulation software are necessary. However, no details about the adjustments are mentioned in the existing studies.