Stability and Convergence of Stochastic Approximation using the O.D.E. Method

Borkar, VS and Meyn, SP (1998) Stability and Convergence of Stochastic Approximation using the O.D.E. Method. In: 37th IEEE Conference on Decision and Control, 1998, 16-18 December, Florida,USA, Vol.1, 277-282.

Preview

PDF stability.pdf Download (512kB)

Abstract

It is shown that the stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated ODE. This in turn implies convergence of the algorithm. Several specific classes of algorithms are considered as applications. It is found that the results provide: (i) a simpler derivation of known results for reinforcement learning algorithms; (ii) a proof for the first time that a class of asynchronous stochastic approximation algorithms are convergent without using any a priori assumption of stability; and (iii) a proof for the first time that asynchronous adaptive critic and Q-learning algorithms are convergent for the average cost optimal control problem.

Item Type:	Conference Paper
Publisher:	IEEE
Additional Information:	Copyright 1990 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	17 Mar 2006
Last Modified:	23 Feb 2019 04:37
URI:	http://eprints.iisc.ac.in/id/eprint/6096

Actions (login required)

View Item