Detecting an Odd Restless Markov Arm with a Trembling Hand

Karthik, PN and Sundaresan, R (2020) Detecting an Odd Restless Markov Arm with a Trembling Hand. In: 2020 IEEE International Symposium on Information Theory, ISIT 2020, 21-26 July 2020, Los Angeles; United States, pp. 2795-2800.

PDF iee_int_Sym_inf_the_pro_2020_2795-2800_2020.pdf - Published Version Restricted to Registered users only Download (235kB) | Request a copy

Abstract

Consider a multi-armed bandit whose arms are independent Markov processes on a common underlying state space. The transition probability matrix of one of the arms (the odd arm) is different from the common transition probability matrix of all the other arms. The goal is to identify the odd arm as quickly as possible while keeping the probability of decision error small. We study the case of restless Markov observations and identify an asymptotic lower bound on the expected stopping time for a decision with vanishing error probability. We then propose a sequential test and show that the asymptotic behaviour of its expected stopping time comes arbitrarily close to that of the lower bound. Prior works dealt with iid arms and rested Markov arms, whereas our work deals with restless Markov arms.

Item Type:	Conference Paper
Publication:	IEEE International Symposium on Information Theory - Proceedings
Publisher:	Institute of Electrical and Electronics Engineers Inc
Additional Information:	The copyright of this article belongs to Institute of Electrical and Electronics Engineers Inc
Keywords:	Markov processes, Asymptotic behaviour; Decision errors; Error probabilities; Lower bounds; Multi armed bandit; Sequential tests; Stopping time; Transition probability matrix, Information theory
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering Division of Interdisciplinary Sciences > Robert Bosch Centre for Cyber Physical Systems
Date Deposited:	24 Sep 2020 07:27
Last Modified:	24 Sep 2020 07:27
URI:	http://eprints.iisc.ac.in/id/eprint/66614

Actions (login required)

View Item