ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

Bhatnagar, Shalabh and Lakshmanan, K (2012) An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes. In: JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 153 (3). pp. 688-708.

[img] PDF
MAT_STA_153-3_688-708_2012.pdf - Published Version
Restricted to Registered users only

Download (891kB) | Request a copy
Official URL: http://dx.doi.org/10.1007/s10957-012-9989-5

Abstract

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Item Type: Journal Article
Publication: JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS
Publisher: Springer Link
Additional Information: Copy right for this article belongs to Springer Science
Keywords: ASYNCHRONOUS STOCHASTIC APPROXIMATIONS;NETWORKS
Department/Centre: Division of Electrical Sciences > Computer Science & Automation
Date Deposited: 17 Jul 2012 11:22
Last Modified: 17 Jul 2012 11:22
URI: http://eprints.iisc.ac.in/id/eprint/44623

Actions (login required)

View Item View Item