Kumar, Ashok M and Sundaresan, Rajesh
(2015)
*Minimization Problems Based on Relative alpha-Entropy I: Forward Projection.*
In: IEEE TRANSACTIONS ON INFORMATION THEORY, 61
(9).
pp. 5063-5080.

PDF
IEEE_Tra_on_Inf_The_61-9_5063_2015.pdf - Published Version Restricted to Registered users only Download (626kB) | Request a copy |

## Abstract

Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative alpha-entropies (denoted I-alpha), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative alpha-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimizers of these relative alpha-entropies on closed and convex sets are shown to exist. Such minimizations generalize the maximum Renyi or Tsallis entropy principle. The minimizing probability distribution (termed forward I-alpha-projection) for a linear family is shown to obey a power-law. Other results in connection with statistical inference, namely subspace transitivity and iterated projections, are also established. In a companion paper, a related minimization problem of interest in robust statistics that leads to a reverse I-alpha-projection is studied.

Item Type: | Journal Article |
---|---|

Publication: | IEEE TRANSACTIONS ON INFORMATION THEORY |

Publisher: | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |

Additional Information: | Copy right for this article belongs to the IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141 USA |

Keywords: | Best approximant; exponential family; information geometry; Kullback-Leibler divergence; linear family; power-law family; projection; Pythagorean property; relative entropy; Renyi entropy; Tsallis entropy |

Department/Centre: | Division of Electrical Sciences > Electrical Communication Engineering |

Date Deposited: | 24 Sep 2015 04:49 |

Last Modified: | 24 Sep 2015 04:49 |

URI: | http://eprints.iisc.ac.in/id/eprint/52389 |

### Actions (login required)

View Item |