Reinforcement learning with average cost for adaptive control of traffic lights at intersections

Prashanth, LA and Bhatnagar, Shalabh (2011) Reinforcement learning with average cost for adaptive control of traffic lights at intersections. In: 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), 5-7 Oct. 2011, Washington, DC, USA.

PDF Reinforcement_Learning.pdf - Published Version Restricted to Registered users only Download (597kB) | Request a copy

Abstract

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Item Type:	Conference Paper
Publisher:	IEEE
Additional Information:	Copyright 2011 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Keywords:	Traffic signal control;reinforcement learning;Q-learning; policy gradient actor-critic.
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	27 Dec 2011 09:09
Last Modified:	27 Dec 2011 09:09
URI:	http://eprints.iisc.ac.in/id/eprint/42822

Actions (login required)

View Item