Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

Prashanth, LA and Chatterjee, Abhranil and Bhatnagar, Shalabh (2014) Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks. In: 6th International Conference on Communication Systems and Networks (COMSNETS), JAN 07-10, 2014, Bangalore, INDIA.

PDF 6th_int_con_com_sys_net_2014.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Item Type:	Conference Proceedings
Series.:	International Conference on Communication Systems and Networks
Publisher:	IEEE
Additional Information:	6th International Conference on Communication Systems and Networks (COMSNETS), Bangalore, INDIA, JAN 07-10, 2014
Keywords:	Sensor Networks; Sleep-Wake Scheduling; Reinforcement Learning; Q-learning; Function Approximation; SPSA
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation Division of Electrical Sciences > Electrical Engineering
Date Deposited:	21 Apr 2015 07:41
Last Modified:	06 May 2015 09:26
URI:	http://eprints.iisc.ac.in/id/eprint/51339

Actions (login required)

View Item