Minimization Problems Based on Relative alpha-Entropy II: Reverse Projection

Kumar, Ashok M and Sundaresan, Rajesh (2015) Minimization Problems Based on Relative alpha-Entropy II: Reverse Projection. In: IEEE TRANSACTIONS ON INFORMATION THEORY, 61 (9). pp. 5081-5095.

PDF IEEE_Tra_on_Tnf_The_61-9_5081_2015.pdf - Published Version Restricted to Registered users only Download (467kB) | Request a copy

Abstract

In part I of this two-part work, certain minimization problems based on a parametric family of relative entropies (denoted I-alpha) were studied. Such minimizers were called forward I-alpha-projections. Here, a complementary class of minimization problems leading to the so-called reverse I-alpha-projections are studied. Reverse I-alpha-projections, particularly on log-convex or power-law families, are of interest in robust estimation problems (alpha > 1) and in constrained compression settings (alpha < 1). Orthogonality of the power-law family with an associated linear family is first established and is then exploited to turn a reverse I-alpha-projection into a forward I-alpha-projection. The transformed problem is a simpler quasi-convex minimization subject to linear constraints.

Item Type:	Journal Article
Publication:	IEEE TRANSACTIONS ON INFORMATION THEORY
Publisher:	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Additional Information:	Copy right for this article belongs to the IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141 USA
Keywords:	Best approximant; exponential family; information geometry; Kullback-Leibler divergence; linear family; power-law family; projection; Pythagorean property; relative entropy; Renyi entropy; robust estimation; Tsallis entropy
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	24 Sep 2015 04:50
Last Modified:	24 Sep 2015 04:50
URI:	http://eprints.iisc.ac.in/id/eprint/52390

Actions (login required)

View Item