Collaborative Learning of Stochastic Bandits Over a Social Network

Kolla, Ravi Kumar and Jagannathan, Krishna and Gopalan, Aditya (2018) Collaborative Learning of Stochastic Bandits Over a Social Network. In: IEEE-ACM TRANSACTIONS ON NETWORKING, 26 (4). pp. 1782-1795.

PDF Ieee_ACM_Tra_Net_26-4_1782_2018.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

We consider a collaborative online learning paradigm, wherein a group of agents connected through a social network are engaged in learning a stochastic multi-armed bandit problem. Each time an agent takes an action, the corresponding reward is instantaneously observed by the agent, as well as its neighbors in the social network. We perform a regret analysis of various policies in this collaborative learning setting. A key finding of this paper is that natural extensions of widely studied single agent learning policies to the network setting need not perform well in terms of regret. In particular, we identify a class of non-altruistic and individually consistent policies and argue by deriving regret lower bounds that they are liable to suffer a large regret in the networked setting. We also show that the learning performance can be substantially improved if the agents exploit the structure of the network and develop a simple learning algorithm based on dominating sets of the network. Specifically, we first consider a star network, which is a common motif in hierarchical social networks and show analytically that the hub agent can be used as an information sink to expedite learning and improve the overall regret. We also derive network-wide regret bounds for the algorithm applied to general networks. We conduct numerical experiments on a variety of networks to corroborate our analytical results.

Item Type:	Journal Article
Publication:	IEEE-ACM TRANSACTIONS ON NETWORKING
Publisher:	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Additional Information:	Copy right for this article belong to IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141 USA
Keywords:	Online learning; multi armed bandits; regret; dominating set
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	12 Sep 2018 15:17
Last Modified:	12 Sep 2018 15:17
URI:	http://eprints.iisc.ac.in/id/eprint/60662

Actions (login required)

View Item