Naskar, A and Thoppe, G (2023) Convergence of Momentum-based Distributed Stochastic Approximation with RL Applications. In: UNSPECIFIED, pp. 178-179.
![]() |
PDF
9th_Ind_con_con_2023. pdf - Published Version Restricted to Registered users only Download (266kB) | Request a copy |
Abstract
We develop a novel proof strategy for deriving almost sure convergence of momentum-based distributed stochas-Tic approximation (DSA) schemes. Popular momentum-based schemes such as Polyak's heavy-ball and Nesterov's Accelerated SGD can be analyzed using our template. Our technique enables us to do away with three restrictive assumptions of existing approaches. One, we do not need the communication matrix to be doubly stochastic. Two, we do not need the noise to be uniformly bounded. Lastly, our approach can handle cases where there are multiple or non-point attractors. As an application, we use our technique to derive convergence for momentum-based extensions of the multi-Agent TD(O) algorithm, where the above restrictive assumptions do not hold. © 2023 IEEE.
Item Type: | Conference Paper |
---|---|
Publication: | 2023 9th Indian Control Conference, ICC 2023 - Proceedings |
Publisher: | Institute of Electrical and Electronics Engineers Inc. |
Additional Information: | The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc. |
Keywords: | Multi agent systems; Stochastic systems, Almost sure convergence; Approximation scheme; Communication matrixes; Doubly stochastic; Multi agent; O algorithm; Proof strategy; Stochastic approximations; Uniformly bounded, Momentum |
Department/Centre: | Division of Electrical Sciences > Computer Science & Automation |
Date Deposited: | 16 May 2024 05:22 |
Last Modified: | 16 May 2024 05:22 |
URI: | https://eprints.iisc.ac.in/id/eprint/84511 |
Actions (login required)
![]() |
View Item |