Simulation-Based Optimization Algorithms for Finite-Horizon Markov Decision Processes

Bhatnagar, Shalabh and Abdulla, Mohammed Shahid (2008) Simulation-Based Optimization Algorithms for Finite-Horizon Markov Decision Processes. In: Simulation- Transactions of the Society for Modeling and Simulation international, 84 (12). pp. 577-600.

Preview

PDF 577.pdf Download (518kB)

Abstract

We develop four simulation-based algorithms for finite-horizon Markov decision processes. Two of these algorithms are developed for finite state and compact action spaces while the other two are for finite state and finite action spaces. Of the former two, one algorithm uses a linear parameterization for the policy, resulting in reduced memory complexity. Convergence analysis is briefly sketched and illustrative numerical experiments with the four algorithms are shown for a problem of flow control in communication networks.

Item Type:	Journal Article
Publication:	Simulation- Transactions of the Society for Modeling and Simulation international
Publisher:	Sage Publications
Additional Information:	Copyright of this article belongs to Sage Publications.
Keywords:	Finite-horizon Markov decision processes;simulation-based algorithmstwo-timescale stochastic approximation;function approximation;actor-critic algorithms;normalized Hadamard matrices.
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	09 Jul 2009 07:51
Last Modified:	22 Feb 2012 06:53
URI:	http://eprints.iisc.ac.in/id/eprint/17467

Actions (login required)

View Item