A stability criterion for two timescale stochastic approximation schemes

Lakshminarayanan, Chandrashekar and Bhatnagar, Shalabh (2017) A stability criterion for two timescale stochastic approximation schemes. In: AUTOMATICA, 79 . pp. 108-114.

PDF Aut_79_108_2017.pdf - Published Version Restricted to Registered users only Download (584kB) | Request a copy

Abstract

We present the first sufficient conditions that guarantee stability of two-timescale stochastic approximation schemes. Our analysis is based on the ordinary differential equation (ODE) method and is an extension of the results in Borkar and Meyn (2000) for single-timescale schemes. As an application of our result, we show the stability of iterates in a two-timescale stochastic approximation scheme arising in reinforcement learning. (C) 2016 Published by Elsevier Ltd.

Item Type:	Journal Article
Publication:	AUTOMATICA
Publisher:	PERGAMON-ELSEVIER SCIENCE LTD, THE BOULEVARD, LANGFORD LANE, KIDLINGTON, OXFORD OX5 1GB, ENGLAND
Additional Information:	Copy right for this article belongs to the PERGAMON-ELSEVIER SCIENCE LTD, THE BOULEVARD, LANGFORD LANE, KIDLINGTON, OXFORD OX5 1GB, ENGLAND
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	20 May 2017 06:02
Last Modified:	20 May 2017 06:02
URI:	http://eprints.iisc.ac.in/id/eprint/56934

Actions (login required)

View Item