Learning with subquadratic regularization: A primal-dual approach

Sankaran, R and Bach, F and Bhattacharyya, C (2020) Learning with subquadratic regularization: A primal-dual approach. In: IJCAI International Joint Conference on Artificial Intelligence, 1 January 2021, Yokohama, pp. 1963-1969.

PDF IJCAI_2020.pdf - Published Version Restricted to Registered users only Download (261kB) | Request a copy

Abstract

Subquadratic norms have been studied recently in the context of structured sparsity, which has been shown to be more beneficial than conventional regularizers in applications such as image denoising, compressed sensing, banded covariance estimation, etc. While existing works have been successful in learning structured sparse models such as trees, graphs, their associated optimization procedures have been inefficient because of hard-to-evaluate proximal operators of the norms. In this paper, we study the computational aspects of learning with subquadratic norms in a general setup. Our main contributions are two proximal-operator based algorithms ADMM-? and CP-?, which generically apply to these learning problems with convex loss functions, and achieve a proven rate of convergence of O(1/T) after T iterations. These algorithms are derived in a primal-dual framework, which have not been examined for subquadratic norms. We illustrate the efficiency of the algorithms developed in the context of tree-structured sparsity, where they comprehensively outperform relevant baselines. © 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved.

Item Type:	Conference Paper
Publication:	IJCAI International Joint Conference on Artificial Intelligence
Publisher:	International Joint Conferences on Artificial Intelligence
Additional Information:	The copyright for this article belongs to International Joint Conferences on Artificial Intelligence.
Keywords:	Forestry; Image denoising; Trees (mathematics), Computational aspects; Covariance estimation; Learning problem; Optimization procedures; Primal-dual approach; Rate of convergence; Structured sparsities; Tree-structured, Artificial intelligence
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	07 Feb 2023 09:33
Last Modified:	07 Feb 2023 09:33
URI:	https://eprints.iisc.ac.in/id/eprint/80000

Actions (login required)

View Item