The RBMLE method for Reinforcement Learning

Mete, A and Singh, R and Kumar, PR (2022) The RBMLE method for Reinforcement Learning. In: 56th Annual Conference on Information Sciences and Systems, CISS 2022, 9-11 March 2022, Princeton, pp. 107-112.

PDF CISS_107-112_2022.pdf - Published Version Restricted to Registered users only Download (804kB) | Request a copy

Abstract

The Reward Biased Maximum Likelihood Estimate (RBMLE) method was proposed about four decades ago for the adaptive control of unknown Markov Decision Processes, and later studied for more general Controlled Markovian Systems and Linear Quadratic Gaussian systems. It showed that if one could bias the Maximum Likelihood Estimate in favor of parameters with larger rewards then one could obtain long-term average optimality. It provided a reason for preferring parameters with larger rewards based on the fact that generally one can only identify the behavior of a system under closed-loop, and therefore any limiting parameter estimate has to necessarily have lower reward than the true parameter. It thereby provided a reason for what his now called 'optimism in the face of uncertainty'. It similarly preceded the definition of 'regret', and it is only in the last three years that it has been analyzed for its regret performance, both analytically, and in comparative simulation testing. This paper provides an account of the RBMLE method for reinforcement learning. © 2022 IEEE.

Item Type:	Conference Proceedings
Publication:	2022 56th Annual Conference on Information Sciences and Systems, CISS 2022
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Adaptive control systems; Learning algorithms; Markov processes; Reinforcement learning, Adaptive Control; Gaussian systems; Linear quadratic Gaussian; LQG system; Markov Decision Processes; Markovian; Maximum-likelihood estimate; MDP; Multiarmed bandits (MABs); Reinforcement learnings, Maximum likelihood estimation
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	19 May 2022 06:54
Last Modified:	19 May 2022 06:54
URI:	https://eprints.iisc.ac.in/id/eprint/72052

Actions (login required)

View Item