Francis, R and Chepuri, SP (2023) Decentralized Stochastic Projection-Free Learning with Compressed Push-Sum. In: UNSPECIFIED.
PDF
Iee_int_wor_mac_lea_sig_pro_mls_2023 - Published Version Restricted to Registered users only Download (277kB) | Request a copy |
Abstract
We consider decentralized stochastic learning methods with data being distributed among multiple nodes. The nodes communicate gradients with their connected neighbors. To reduce communication overhead, several compression techniques have been proposed for solving unconstrained optimization problems in a decentralized setting. In this paper, we focus on decentralized stochastic learning with convex constraints. Specifically, we propose a novel communication-efficient decentralized stochastic Frank-Wolfe algorithm to solve finite-sum constrained minimization problems by communicating only a compressed version of the gradient. The proposed method guarantees a convergence rate of about O(n-1/2k-1/3) for convex objectives, and for non-convex objectives, it guarantees a convergence rate of about O(n-1/2k-2/9), where n is the number of nodes in the network. We empirically validate the efficacy of the proposed algorithm in terms of communication overhead and suboptimality gap on several benchmark machine learning tasks. © 2023 IEEE.
Item Type: | Conference Paper |
---|---|
Publication: | IEEE International Workshop on Machine Learning for Signal Processing, MLSP |
Publisher: | IEEE Computer Society |
Additional Information: | The copyright for this article belongs to IEEE Computer Society. |
Keywords: | Constrained optimization; Learning systems; Machine learning, Communication overheads; Compression techniques; Convergence rates; Convex objectives; Decentralised; Learning methods; Multiple nodes; Stochastic learning; Stochastics; Unconstrained optimization problems, Stochastic systems |
Department/Centre: | Division of Electrical Sciences > Electrical Communication Engineering |
Date Deposited: | 04 Mar 2024 09:13 |
Last Modified: | 04 Mar 2024 09:13 |
URI: | https://eprints.iisc.ac.in/id/eprint/84302 |
Actions (login required)
View Item |