Thoppe, G and Kumar, B (2021) A Law of Iterated Logarithm for Multi-Agent Reinforcement Learning. In: 7th Indian Control Conference, ICC 2021, 20 - 22 December 2021, Mumbai, India, pp. 19-20.
|
PDF
ICC_2021.pdf - Published Version Download (225kB) | Preview |
Abstract
In Multi-Agent Reinforcement Learning (MARL), multiple agents interact with a common environment, as also with each other, for solving a shared problem in sequential decision-making. In this work, we derive a novel law of iterated logarithm for a family of distributed nonlinear stochastic approximation schemes that is useful in MARL. In particular, our result describes the convergence rate on almost every sample path where the algorithm converges. This result is the first of its kind in the distributed setup and provides deeper insights than the existing ones, which only discuss convergence rates in the expected or the CLT sense. Importantly, our result holds under significantly weaker assumptions: neither the gossip matrix needs to be doubly stochastic nor the stepsizes square summable.
| Item Type: | Conference Paper |
|---|---|
| Publication: | 2021 7th Indian Control Conference, ICC 2021 - Proceedings |
| Publisher: | Institute of Electrical and Electronics Engineers Inc. |
| Additional Information: | The copyright for this article belongs to the Authors. |
| Keywords: | Approximation theory; Decision making; Fertilizers; Reinforcement learning; Stochastic systems, Approximation scheme; Common environment; Convergence rates; Distributed setups; Law of iterated logarithm; Multi-agent reinforcement learning; Multiple agents; Sample path; Sequential decision making; Stochastic approximations, Multi agent systems |
| Department/Centre: | Division of Electrical Sciences > Computer Science & Automation |
| Date Deposited: | 23 May 2023 03:51 |
| Last Modified: | 23 May 2023 03:51 |
| URI: | https://eprints.iisc.ac.in/id/eprint/81718 |
Actions (login required)
 |
View Item |
