On the Convergence of a Bayesian Algorithm for Joint Dictionary Learning and Sparse Recovery

Joseph, G and Murthy, CR (2020) On the Convergence of a Bayesian Algorithm for Joint Dictionary Learning and Sparse Recovery. In: IEEE Transactions on Signal Processing, 68 . pp. 343-358.

PDF ieee_tra_sin_pro_68_343-358_2020.pdf - Published Version Restricted to Registered users only Download (967kB) | Request a copy

Abstract

Dictionary learning (DL) is a well-researched problem, where the goal is to learn a dictionary from a finite set of noisy training signals, such that the training data admits a sparse representation over the dictionary. While several solutions are available in the literature, relatively little is known about their convergence and optimality properties. In this paper, we make progress on this problem by analyzing a Bayesian algorithm for DL. Specifically, we cast the DL problem into the sparse Bayesian learning (SBL) framework by imposing a hierarchical Gaussian prior on the sparse vectors. This allows us to simultaneously learn the dictionary as well as the parameters of the prior on the sparse vectors using the expectation-maximization algorithm. The dictionary update step turns out to be a non-convex optimization problem, and we present two solutions, namely, an alternating minimization (AM) procedure and an Armijo line search (ALS) method. We analytically show that the ALS procedure is globally convergent, and establish the stability of the solution by characterizing its limit points. Further, we prove the convergence and stability of the overall DL-SBL algorithm, and show that the minima of the cost function of the overall algorithm are achieved at sparse solutions. As a concrete example, we consider the application of the SBL-based DL algorithm to image denoising, and demonstrate the efficacy of the algorithm relative to existing DL algorithms.

Item Type:	Journal Article
Publication:	IEEE Transactions on Signal Processing
Publisher:	IEEE
Additional Information:	Copyright of this article belongs to IEEE
Keywords:	Convex optimization; Cost functions; Image segmentation; Maximum principle, Alternating minimization; Armijo line searches; Convergence and stability; Dictionary learning; Expectation-maximization algorithms; Nonconvex optimization; Sparse Bayesian learning (SBL); Sparse representation, Image denoising
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	17 Feb 2020 09:22
Last Modified:	17 Feb 2020 09:22
URI:	http://eprints.iisc.ac.in/id/eprint/64537

Actions (login required)

View Item