Scalable sequential alternating proximal methods for sparse structural SVMs and CRFs

Balamurugan, P and Shevade, Shirish and Babu, Ravindra T (2014) Scalable sequential alternating proximal methods for sparse structural SVMs and CRFs. In: KNOWLEDGE AND INFORMATION SYSTEMS, 38 (3). pp. 599-621.

PDF kno_inf_sys_38-3_599_2014.pdf - Published Version Restricted to Registered users only Download (501kB) | Request a copy

Abstract

Structural Support Vector Machines (SSVMs) and Conditional Random Fields (CRFs) are popular discriminative methods used for classifying structured and complex objects like parse trees, image segments and part-of-speech tags. The datasets involved are very large dimensional, and the models designed using typical training algorithms for SSVMs and CRFs are non-sparse. This non-sparse nature of models results in slow inference. Thus, there is a need to devise new algorithms for sparse SSVM and CRF classifier design. Use of elastic net and L1-regularizer has already been explored for solving primal CRF and SSVM problems, respectively, to design sparse classifiers. In this work, we focus on dual elastic net regularized SSVM and CRF. By exploiting the weakly coupled structure of these convex programming problems, we propose a new sequential alternating proximal (SAP) algorithm to solve these dual problems. This algorithm works by sequentially visiting each training set example and solving a simple subproblem restricted to a small subset of variables associated with that example. Numerical experiments on various benchmark sequence labeling datasets demonstrate that the proposed algorithm scales well. Further, the classifiers designed are sparser than those designed by solving the respective primal problems and demonstrate comparable generalization performance. Thus, the proposed SAP algorithm is a useful alternative for sparse SSVM and CRF classifier design.

Item Type:	Journal Article
Publication:	KNOWLEDGE AND INFORMATION SYSTEMS
Publisher:	SPRINGER LONDON LTD
Additional Information:	Copyright for this article belongs to the SPRINGER LONDON LTD, ENGLAND
Keywords:	Structural SVM; CRF; Elastic net; Sequential alternating proximal method
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	11 Apr 2014 10:05
Last Modified:	11 Apr 2014 10:06
URI:	http://eprints.iisc.ac.in/id/eprint/48846

Actions (login required)

View Item