Two-Timescale Simultaneous Perturbation Stochastic Approximation Using Deterministic Perturbation Sequences

Bhatnagar, Shalabh and Fu, Michael C and Marcus, Steven I and Wang, I-Jeng (2003) Two-Timescale Simultaneous Perturbation Stochastic Approximation Using Deterministic Perturbation Sequences. In: ACM Transactions on Modeling and Computer Simulation, 13 (2). pp. 180-209.

Preview

PDF p180-bhatnagar.pdf Download (293kB)

Abstract

Simultaneous perturbation stochastic approximation (SPSA) algorithms have been found to be very effective for high-dimensional simulation optimization problems. The main idea is to estimate the gradient using simulation output performance measures at only two settings of the N-dimensional parameter vector being optimized rather than at the N + 1 or 2N settings required by the usual one-sided or symmetric difference estimates, respectively. The two settings of the parameter vector are obtained by simultaneously changing the parameter vector in each component direction using random perturbations. In this article, in order to enhance the convergence of these algorithms, we consider deterministic sequences of perturbations for two-timescale SPSA algorithms. Two constructions for the perturbation sequences are considered: complete lexicographical cycles and much shorter sequences based on normalized Hadamard matrices. Recently, one-simulation versions of SPSA have been proposed, and we also investigate these algorithms using deterministic sequences. Rigorous convergence analyses for all proposed algorithms are presented in detail. Extensive numerical experiments on a network of M/G/1 queues with feedback indicate that the deterministic sequence SPSA algorithms perform significantly better than the corresponding randomized algorithms.

Item Type:	Journal Article
Publication:	ACM Transactions on Modeling and Computer Simulation
Publisher:	ACM Press
Additional Information:	©ACM,2003.This is the author's version of the work.It is posted here by permission of ACM for your personal use.Not for redistribution.The definitive version was published in ACM Transactions on Modeling and Computer Simulation, Vol. 13, No. 2, April 2003. http://doi.acm.org/10.1145/858481.858486
Keywords:	Probabilistic Algorithms;Simulation Theory;Algorithms;Performance;Theory;Simulation optimization;stochastic approximation;SPSA;two-timescale algorithms;deterministic perturbations;Hadamard matrices
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	08 Jun 2004
Last Modified:	19 Sep 2010 04:12
URI:	http://eprints.iisc.ac.in/id/eprint/266

Actions (login required)

View Item