Adaptive System Optimization Using Random Directions Stochastic Approximation

Prashanth, L A and Bhatnagar, Shalabh and Fu, Michael and Marcus, Steve (2017) Adaptive System Optimization Using Random Directions Stochastic Approximation. In: IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 62 (5). pp. 2223-2238.

PDF Iee_Tra_Aut_Con_62-5_2223_2017.pdf - Published Version Restricted to Registered users only Download (606kB) | Request a copy

Abstract

We present new algorithms for simulation optimization using random directions stochastic approximation (RDSA). These include first-order (gradient) as well as second-order (Newton) schemes. We incorporate both continuous-valued as well as discrete-valued perturbations into both types of algorithms. The former are chosen to be independent and identically distributed (i.i.d.) symmetric uniformly distributed random variables (r.v.), while the latter are i.i.d. asymmetric Bernoulli r.v.s. Our Newton algorithm, with a novel Hessian estimation scheme, requires N-dimensional perturbations and three loss measurements per iteration, whereas the simultaneous perturbation Newton search algorithm of 1] requires 2N-dimensional perturbations and four loss measurements per iteration. We prove the asymptotic unbiasedness of both gradient and Hessian estimates and asymptotic (strong) convergence for both first-order and second-order schemes. We also provide asymptotic normality results, which in particular establish that the asymmetric Bernoulli variant of Newton RDSA method is better than 2SPSA of 1]. Numerical experiments are used to validate the theoretical results.

Item Type:	Journal Article
Publication:	IEEE TRANSACTIONS ON AUTOMATIC CONTROL
Additional Information:	Copy right for this article belongs to the IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141 USA
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	03 Jun 2017 09:36
Last Modified:	03 Jun 2017 09:36
URI:	http://eprints.iisc.ac.in/id/eprint/57094

Actions (login required)

View Item