Effective Memory Shrinkage in Estimation

Jain, Ayush and Tyagi, Himanshu (2018) Effective Memory Shrinkage in Estimation. In: IEEE International Symposium on Information Theory (ISIT), JUN 17-22, 2018, Vail, CO, pp. 1071-1075.

PDF ISIT_1071_2018.pdf - Published Version Restricted to Registered users only Download (700kB) | Request a copy

Abstract

It is known that a processor with limited memory consisting of an m-state machine can distinguish two coins with biases that differ by 1/m. On the other hand, the best additive accuracy with which the same processor can estimate the bias of a coin is only 1/root m. We demystify this apparent shrinkage in memory by showing that for any such estimator using an m-state machine, there exist two values of the bias that are 1/root m apart but for which the effective number of states available to resolve them is only O(root m). Building on this result, we show that the number of bits of memory required to estimate a bias in the interval (a; a2(alpha)) with a multiplicative accuracy of 2(+/-delta) is log(alpha/delta(2)), up to an additive constant. In fact, we show that the lower bound is attained by a Gaussian counter, namely a probabilistic counter whose stationary distribution has a Gaussian form. This gives a precise characterization of memory-complexity of bias estimation along with a heuristically appealing family of optimal estimators. Underlying our results are new bounds for estimation of the natural parameter of a discrete exponential family, which maybe of independent interest.

Item Type:	Conference Proceedings
Series.:	IEEE International Symposium on Information Theory
Publisher:	IEEE
Additional Information:	Copy right for this article belong to IEEE
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	22 Nov 2018 15:03
Last Modified:	22 Nov 2018 15:03
URI:	http://eprints.iisc.ac.in/id/eprint/61131

Actions (login required)

View Item