Structured Learning for Non-Smooth Ranking losses

Chakrabarti, Soumen and Khanna, Rajiv and Sawant, Uma and Bhattacharyya, Chiru (2008) Structured Learning for Non-Smooth Ranking losses. In: In KDD.

PDF Structured.pdf - Published Version Restricted to Registered users only Download (470kB) | Request a copy

Abstract

Learning to rank from relevance judgment is an active research area. Itemwise score regression, pairwise preference satisfaction, and listwise structured learning are the major techniques in use. Listwise structured learning has been applied recently to optimize important non-decomposable ranking criteria like AUC (area under ROC curve) and MAP(mean average precision). We propose new, almost-lineartime algorithms to optimize for two other criteria widely used to evaluate search systems: MRR (mean reciprocal rank) and NDCG (normalized discounted cumulative gain)in the max-margin structured learning framework. We also demonstrate that, for different ranking criteria, one may need to use different feature maps. Search applications should not be optimized in favor of a single criterion, because they need to cater to a variety of queries. E.g., MRR is best for navigational queries, while NDCG is best for informational queries. A key contribution of this paper is to fold multiple ranking loss functions into a multi-criteria max-margin optimization.The result is a single, robust ranking model that is close to the best accuracy of learners trained on individual criteria. In fact, experiments over the popular LETOR and TREC data sets show that, contrary to conventional wisdom, a test criterion is often not best served by training with the same individual criterion.

Item Type:	Conference Paper
Keywords:	Max-margin structured learning to rank;Nondecomposable loss functions.
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	23 Sep 2011 09:18
Last Modified:	23 Sep 2011 09:18
URI:	http://eprints.iisc.ac.in/id/eprint/40706

Actions (login required)

View Item