Multi-agent Reinforcement Learning for Traffic Signal Control

Prabuchandran, KJ and Kumar, Hemanth AN and Bhatnagar, Shalabh (2014) Multi-agent Reinforcement Learning for Traffic Signal Control. In: IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), OCT 08-11, 2014, Qingdao, PEOPLES R CHINA, pp. 2529-2534.

PDF 2014_IEEE_17th_Int_Con_on_Int_Tra_Sys_2529_2014.pdf - Published Version Restricted to Registered users only Download (365kB) | Request a copy

Abstract

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Item Type:	Conference Proceedings
Publisher:	IEEE
Additional Information:	Copy right for this article belongs to the IEEE, 345 E 47TH ST, NEW YORK, NY 10017 USA
Keywords:	traffic signal control; multi-agent reinforcement learning; Q-learning; UCB; VISSIM
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	19 Aug 2015 07:14
Last Modified:	19 Aug 2015 07:14
URI:	http://eprints.iisc.ac.in/id/eprint/52166

Actions (login required)

View Item