ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

A Model based Search Method for Prediction in Model-free Markov Decision Process

Joseph, Ajin George and Bhatnagar, Shalabh (2017) A Model based Search Method for Prediction in Model-free Markov Decision Process. In: International Joint Conference on Neural Networks (IJCNN), MAY 14-19, 2017, Anchorage, AK, pp. 170-177.

[img] PDF
Int_Joi_Con_Neu_Two_170_2017.pdf - Published Version
Restricted to Registered users only

Download (356kB) | Request a copy
Official URL: http://dx.doi.org/10.1109/IJCNN.2017.7965851


In this paper, we provide a new algorithm for the problem of prediction in the model-free MDP setting, i.e., estimating the value function of a given policy using the linear function approximation architecture, with memory and computation costs scaling quadratically in the size of the feature set. The algorithm is a multi-timescale variant of the very popular cross entropy (CE) method which is a model based search method to find the global optimum of a real-valued function. This is the first time a model based search method is used for the prediction problem. A proof of convergence using the ODE method is provided. The theoretical results are supplemented with experimental comparisons. The algorithm achieves good performance fairly consistently on many benchmark problems.

Item Type: Conference Proceedings
Additional Information: Copy right for the article belong to IEEE, 345 E 47TH ST, NEW YORK, NY 10017 USA
Department/Centre: Division of Interdisciplinary Research > Supercomputer Education & Research Centre
Depositing User: Id for Latest eprints
Date Deposited: 13 Apr 2018 19:56
Last Modified: 23 Oct 2018 14:48
URI: http://eprints.iisc.ac.in/id/eprint/59552

Actions (login required)

View Item View Item