ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Sequential Multi-Hypothesis Testing in Multi-Armed Bandit Problems: An Approach for Asymptotic Optimality

Prabhu, GR and Bhashyam, S and Gopalan, A and Sundaresan, R (2022) Sequential Multi-Hypothesis Testing in Multi-Armed Bandit Problems: An Approach for Asymptotic Optimality. In: IEEE Transactions on Information Theory, 68 (7). pp. 4790-4817.

IEEE_tra_inf_the_68-7_4790-4817_2022.pdf - Published Version

Download (1MB) | Preview
Official URL: https://doi.org/10.1109/TIT.2022.3159600


We consider a multi-hypothesis testing problem involving a K -armed bandit. Each arm's signal follows a distribution from a vector exponential family. The actual parameters of the arms are unknown to the decision maker. The decision maker incurs a delay cost for delay until a decision and a switching cost whenever he switches from one arm to another. His goal is to minimise the overall cost until a decision is reached on the true hypothesis. Of interest are policies that satisfy a given constraint on the probability of false detection. This is a sequential decision making problem where the decision maker gets only a limited view of the true state of nature at each stage, but can control his view by choosing the arm to observe at each stage. An information-theoretic lower bound on the total cost (expected time for a reliable decision plus total switching cost) is first identified, and a variation on a sequential policy based on the generalised likelihood ratio statistic is then studied. Due to the vector exponential family assumption, the signal processing at each stage is simple; the associated conjugate prior distribution on the unknown model parameters enables easy updates of the posterior distribution. The proposed policy, with a suitable threshold for stopping, is shown to satisfy the given constraint on the probability of false detection. Under a continuous selection assumption, the policy is also shown to be asymptotically optimal in terms of the total cost among all policies that satisfy the constraint on the probability of false detection.

Item Type: Journal Article
Publication: IEEE Transactions on Information Theory
Publisher: Institute of Electrical and Electronics Engineers Inc.
Additional Information: The copyright for this article belongs to the Author(s).
Keywords: Decision making; Probability; Signal processing; Statistics; Switching, Action planning; Active Sensing; Conjugate prior; Delay; Exponential family; Hypothesis testing; Multiarmed bandits (MABs); Optimisations; Relative entropy; Search problem; Sequential analysis; Switching costs, Cost benefit analysis
Department/Centre: Division of Electrical Sciences > Electrical Communication Engineering
Division of Interdisciplinary Sciences > Robert Bosch Centre for Cyber Physical Systems
Date Deposited: 19 Sep 2022 10:15
Last Modified: 19 Sep 2022 10:15
URI: https://eprints.iisc.ac.in/id/eprint/76611

Actions (login required)

View Item View Item