Classification calibration dimension for general multiclass losses

Ramaswamy, Harish G and Agarwal, Shivani (2012) Classification calibration dimension for general multiclass losses. In: 2012 In Advances in Neural Information Processing Systems (NIPS), December 05, 2012, Harveys Convention Center Floor, CC.

PDF Adva_Neural Information Processing Systems_1_2012.pdf - Published Version Restricted to Registered users only Download (496kB) | Request a copy

Abstract

We study consistency properties of surrogate loss functions for general multiclass classification problems, defined by a general loss matrix. We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning problems), to the general multiclass setting, and derive necessary and sufficient conditions for a surrogate loss to be classification calibrated with respect to a loss matrix in this setting. We then introduce the notion of \emph{classification calibration dimension} of a multiclass loss matrix, which measures the smallest `size' of a prediction space for which it is possible to design a convex surrogate that is classification calibrated with respect to the loss matrix. We derive both upper and lower bounds on this quantity, and use these results to analyze various loss matrices. In particular, as one application, we provide a different route from the recent result of Duchi et al.\ (2010) for analyzing the difficulty of designing `low-dimensional' convex surrogates that are consistent with respect to pairwise subset ranking losses. We anticipate the classification calibration dimension may prove to be a useful tool in the study and design of surrogate losses for general multiclass learning problems.

Item Type:	Conference Paper
Publisher:	Neural Information Processing Systems
Additional Information:	Copyright of this article belongs to Neural Information Processing Systems.
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	08 Nov 2013 05:18
Last Modified:	08 Nov 2013 05:18
URI:	http://eprints.iisc.ac.in/id/eprint/47726

Actions (login required)

View Item