Variable Sparsity Kernel Learning

Aflalo, Jonathan and Ben-Tal, Aharon and Bhattacharyya, Chiranjib and Nath, Jagarlapudi Saketha and Raman, Sankaran (2011) Variable Sparsity Kernel Learning. In: Journal of Machine Learning Research, 12 . pp. 565-592.

PDF Variable.pdf - Published Version Restricted to Registered users only Download (238kB) | Request a copy

Abstract

This paper(1) presents novel algorithms and applications for a particular class of mixed-norm regularization based Multiple Kernel Learning (MKL) formulations. The formulations assume that the given kernels are grouped and employ l(1) norm regularization for promoting sparsity within RKHS norms of each group and l(s), s >= 2 norm regularization for promoting non-sparse combinations across groups. Various sparsity levels in combining the kernels can be achieved by varying the grouping of kernels-hence we name the formulations as Variable Sparsity Kernel Learning (VSKL) formulations. While previous attempts have a non-convex formulation, here we present a convex formulation which admits efficient Mirror-Descent (MD) based solving techniques. The proposed MD based algorithm optimizes over product of simplices and has a computational complexity of O (m(2)n(tot) log n(max)/epsilon(2)) where m is no. training data points, n(max), n(tot) are the maximum no. kernels in any group, total no. kernels respectively and epsilon is the error in approximating the objective. A detailed proof of convergence of the algorithm is also presented. Experimental results show that the VSKL formulations are well-suited for multi-modal learning tasks like object categorization. Results also show that the MD based algorithm outperforms state-of-the-art MKL solvers in terms of computational efficiency.

Item Type:	Journal Article
Publication:	Journal of Machine Learning Research
Publisher:	MIT Press
Additional Information:	Copyright of this article belongs to MIT Press.
Keywords:	multiple kernel learning;mirror descent;mixed-norm;object categorization;scalability
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	26 Apr 2011 07:58
Last Modified:	26 Apr 2011 07:58
URI:	http://eprints.iisc.ac.in/id/eprint/37152

Actions (login required)

View Item