Stochastic recursive inclusions in two timescales with nonadditive iterate-dependent markov noise

Yaji, VG and Bhatnagar, S (2020) Stochastic recursive inclusions in two timescales with nonadditive iterate-dependent markov noise. In: Mathematics of Operations Research, 45 (4). pp. 1405-1444.

PDF mat_ope_res_45-04_1405-1444_2020 - Published Version Download (1MB)

Abstract

In this paper, we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of nonadditive iterate-dependent Markov noise. We show that the recursion on each timescale tracks the flow of a differential inclusion obtained by averaging the set-valued drift function in the recursion with respect to a set of measures accounting for both averaging with respect to the stationary distributions of the Markov noise terms and the interdependence between the two recursions on different timescales. The framework studied in this paper builds on a recent work by Ramaswamy and Bhatnagar, by allowing for the presence of nonadditive iterate-dependent Markov noise. As an application, we consider the problem of computing the optimum in a constrained convex optimization problem, where the objective function and the constraints are averaged with respect to the stationary distribution of an underlying Markov chain. Further, the proposed scheme neither requires the differentiability of the objective function nor the knowledge of the averaging measure. © 2020 INFORMS.

Item Type:	Journal Article
Publication:	Mathematics of Operations Research
Publisher:	INFORMS Inst.for Operations Res.and the Management Sciences
Additional Information:	The copyright of this article belongs to INFORMS Inst.for Operations Res.and the Management Sciences
Keywords:	Approximation theory; Constrained optimization; Convex optimization; Stochastic systems, Asymptotic behaviors; Constrained convex optimizations; Differentiability; Differential inclusions; Drift functions; Objective functions; Stationary distribution; Stochastic approximations, Markov chains
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	21 Jan 2021 07:22
Last Modified:	21 Jan 2021 07:22
URI:	http://eprints.iisc.ac.in/id/eprint/67329

Actions (login required)

View Item