AN ACTOR-CRITIC REINFORCEMENT LEARNING APPROACH IN MULTI-AGENT SYSTEMS FOR URBAN TRAFFIC CONTROL

Q: How can I download an article?

To download an article from SID, first log in to the site, search for the article title, and click on the 'Download Article' option.

Q: How can I download an ISI article?

To download an ISI article on SID, enter the keyword or article title in the search bar, view the relevant results, click on the desired article, and select the 'Download Article' option.

Q: How can I access the SID database?

To access the SID database, visit SID.ir, create an account, and log in to access scientific resources.

Q: Is downloading articles from SID free?

Some articles on SID are available for free, while others require payment. Details are specified on the article's page.

ASLANI M.; MESGARI M.S.; Motieyan h.

مرکز اطلاعات علمی Scientific Information Database (SID) - Trusted Source for Research and Academic Resources

Journal Paper

Paper Information

Journal: JOURNAL OF GEOMATICS SCIENCE AND TECHNOLOGY Year:2016 | Volume:5 | Issue:3 Page(s): 233-245

Download Full-Text

Persian Verion

View:

1,048

Download:

Cites:

Information Journal Paper

Title

AN ACTOR-CRITIC REINFORCEMENT LEARNING APPROACH IN MULTI-AGENT SYSTEMS FOR URBAN TRAFFIC CONTROL

Author(s)

ASLANI M. | MESGARI M.S. | Motieyan h. | Issue Writer Certificate

Keywords

ACTORQ2

CRITICQ2

ADAPTIVE TRAFFIC SIGNAL CONTROLQ2

MULTIQ2

AGENT SYSTEMSQ2

REINFORCEMENT LEARNINGQ2

Abstract

Nowadays, most urban societies have experienced a new phenomenon so-called urban traffic congestion, which is caused by crossing too many vehicles from the same transportation infrastructure at the same time. Traffic congestion has different consequences such as air pollution, decrease in speed, increase in travel time, fuel consumption and even incidents. One of the feasible solutions for bringing off the increase in transportation demand is to improve the existing infrastructure by means of intelligent traffic control systems. From a traffic engineering point of view, a traffic control system consists of physical network, control devices (traffic signals, variable message signs, so forth), the model of transportation demand and control strategy. The focus of this paper is on the latter especially traffic signal control.Traffic signal control can be modeled by MULTI-agent systems perfectly because of its distributed and autonomous nature. In this context, drivers and traffic signals are considered distributed, autonomous and intelligent agents. Besides, due to high complexity arising in urban traffic patterns and nonstationarity of traffic environment, developing an optimized MULTI-agent system by preprogrammed agent’s behavior is most impractical. Therefore, the agents must, instead, discover their knowledge through a learning mechanism by interacting with the environment.Reinforcement Learning (RL) is a promising approach for training the agent in which optimizes its behavior by interacting with the environment. Each time the agent receives information on the current state of the environment, performs an action in its environment, which may changes the state of the environment, and receives a scalar reward that reflects how appropriate the agent’s behavior has been in the past. The function that indicates the action to take in a certain state is called the policy. The goal of RL is to find a policy that maximizes the long-term reward. Several types of RL algorithms have been introduced and they can be divided into three groups: ACTOR-Only, CRITIC-Only and ACTOR-Critic methods.ACTOR-Only methods typically work with a parameterized family of policies over which optimization procedures can be used directly. Often the gradient of the value of a policy with respect to the policy parameters is estimated and then used to improve the policy. The drawback of ACTOR-Only methods is that the increase of performance is harder to estimate when no value function is learned. CRITIC-Only methods are based on the idea to first find the optimal value function and then to derive an optimal policy from this value function. This approach undermines the ability of using continuous actions and thus of finding the true optimum. In this research, ACTOR-Critic REINFORCEMENT LEARNING is applied as a learning method for true ADAPTIVE TRAFFIC SIGNAL CONTROL. ACTOR-Critic method is a temporal difference method that has a separate memory structure to explicitly represent the policy independent of the value function. The policy structure is known as the ACTOR, because it is used to select actions and the CRITIC is a state-value function.In this paper, AIMSUN, which is a microscopic traffic simulator, is used to model traffic environment. AIMSUN models stochastic vehicle flow by employing car-following, Lane Changing and gap acceptance. AIMSUN API was used to construct the state, execute the action, and calculate the signal reward in each traffic light. The state of the each agent is represented by a vector of 1 + P components, where the first component is the phase number and P is the number of entrance streets which goes to intersection. Also, the action of the agent is the duration of the current phase. The immediate reward is defined as the reduction in the total number of cars waiting in all entrance streets. In fact, difference between the total numbers of cars in two successive decision points is used as a signal reward. The REINFORCEMENT LEARNING controller is benchmarked against optimized pretimed control. The results indicate that the ACTOR-Critic controller decreases Queue length, travel time, fuel consumption and air pollution when compared to optimized pretimed controller.

Cites

No record.

References

No record.

Cite

APA: Copy

ASLANI, M., MESGARI, M.S., & Motieyan, h.. (2016). AN ACTOR-CRITIC REINFORCEMENT LEARNING APPROACH IN MULTI-AGENT SYSTEMS FOR URBAN TRAFFIC CONTROL. JOURNAL OF GEOMATICS SCIENCE AND TECHNOLOGY, 5(3), 233-245. SID. https://sid.ir/paper/249511/en

Vancouver: Copy

ASLANI M., MESGARI M.S., Motieyan h.. AN ACTOR-CRITIC REINFORCEMENT LEARNING APPROACH IN MULTI-AGENT SYSTEMS FOR URBAN TRAFFIC CONTROL. JOURNAL OF GEOMATICS SCIENCE AND TECHNOLOGY[Internet]. 2016;5(3):233-245. Available from: https://sid.ir/paper/249511/en

IEEE: Copy

M. ASLANI, M.S. MESGARI, and h. Motieyan, “AN ACTOR-CRITIC REINFORCEMENT LEARNING APPROACH IN MULTI-AGENT SYSTEMS FOR URBAN TRAFFIC CONTROL,” JOURNAL OF GEOMATICS SCIENCE AND TECHNOLOGY, vol. 5, no. 3, pp. 233–245, 2016, [Online]. Available: https://sid.ir/paper/249511/en

Related Journal Papers