ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

A Hidden Markov Restless Multi-armed Bandit Model for Playout Recommendation Systems

Meshram, Rahul and Gopalan, Aditya and Manjunath, D (2017) A Hidden Markov Restless Multi-armed Bandit Model for Playout Recommendation Systems. In: 9th International Conference on Communication Systems and Networks, COMSNETS 2017, 4 January - 8 January 2017, Bengaluru, pp. 335-362.

Full text not available from this repository.
Official URL: https://doi.org/10.1007/978-3-319-67235-9_19


We consider a restless multi-armed bandit (RMAB) in which each arm can be in one of two states, say 0 or 1. Playing an arm generates a unit reward with a probability that depends on the state of the arm. The belief about the state of the arm can be calculated using a Bayesian update after every play. This RMAB has been designed for use in recommendation systems where the user’s preferences depend on the history of recommendations. In this paper we analyse the RMAB by first studying single armed bandit. We show that it is Whittle-indexable and obtain a closed form expression for the Whittle index. For a RMAB to be useful in practice, we need to be able to learn the parameters of the arms. We present Thompson sampling scheme, that learns the parameters of the arms and also illustrate its performance numerically.

Item Type: Conference Paper
Series.: Lecture Notes in Computer Science
Publisher: Springer
Additional Information: The Copyright of this article belongs to the Springer
Keywords: Automated playlist creation systems; Learning; POMDP; Recommendation systems; Restless multi-armed bandit; Artificial intelligence; Computer science; Computers; Bayesian; Closed-form expression; Hidden markov; Restless multi-armed bandit; Thompson samplings; Two-state; Recommender systems
Department/Centre: Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited: 26 May 2022 04:32
Last Modified: 26 May 2022 04:32
URI: https://eprints.iisc.ac.in/id/eprint/72605

Actions (login required)

View Item View Item