Prabhu, GR and Bhashyam, S and Gopalan, A and Sundaresan, R (2022) Sequential Multi-Hypothesis Testing in Multi-Armed Bandit Problems: An Approach for Asymptotic Optimality. In: IEEE Transactions on Information Theory, 68 (7). pp. 4790-4817.
|
PDF
IEEE_tra_inf_the_68-7_4790-4817_2022.pdf - Published Version Download (1MB) | Preview |
Abstract
We consider a multi-hypothesis testing problem involving a K -armed bandit. Each arm's signal follows a distribution from a vector exponential family. The actual parameters of the arms are unknown to the decision maker. The decision maker incurs a delay cost for delay until a decision and a switching cost whenever he switches from one arm to another. His goal is to minimise the overall cost until a decision is reached on the true hypothesis. Of interest are policies that satisfy a given constraint on the probability of false detection. This is a sequential decision making problem where the decision maker gets only a limited view of the true state of nature at each stage, but can control his view by choosing the arm to observe at each stage. An information-theoretic lower bound on the total cost (expected time for a reliable decision plus total switching cost) is first identified, and a variation on a sequential policy based on the generalised likelihood ratio statistic is then studied. Due to the vector exponential family assumption, the signal processing at each stage is simple; the associated conjugate prior distribution on the unknown model parameters enables easy updates of the posterior distribution. The proposed policy, with a suitable threshold for stopping, is shown to satisfy the given constraint on the probability of false detection. Under a continuous selection assumption, the policy is also shown to be asymptotically optimal in terms of the total cost among all policies that satisfy the constraint on the probability of false detection.
| Item Type: | Journal Article |
|---|---|
| Publication: | IEEE Transactions on Information Theory |
| Publisher: | Institute of Electrical and Electronics Engineers Inc. |
| Additional Information: | The copyright for this article belongs to the Author(s). |
| Keywords: | Decision making; Probability; Signal processing; Statistics; Switching, Action planning; Active Sensing; Conjugate prior; Delay; Exponential family; Hypothesis testing; Multiarmed bandits (MABs); Optimisations; Relative entropy; Search problem; Sequential analysis; Switching costs, Cost benefit analysis |
| Department/Centre: | Division of Electrical Sciences > Electrical Communication Engineering Division of Interdisciplinary Sciences > Robert Bosch Centre for Cyber Physical Systems |
| Date Deposited: | 19 Sep 2022 10:15 |
| Last Modified: | 19 Sep 2022 10:15 |
| URI: | https://eprints.iisc.ac.in/id/eprint/76611 |
Actions (login required)
 |
View Item |
