Actor-critic algorithms for constrained multi-agent reinforcement learning

Diddigi, RB and Prabuchandran, KJ and Sai Koti Reddy, D and Bhatnagar, S (2019) Actor-critic algorithms for constrained multi-agent reinforcement learning. In: 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019, 13 May 2019through 17 May 2019, Montreal, pp. 1931-1933.

PDF AAMAS_2019.pdf - Published Version Restricted to Registered users only Download (294kB) | Request a copy

Abstract

Multi-agent reinforcement learning has gained lot of popularity primarily owing to the success of deep function approximation architectures. However, many real-life multi-agent applications often impose constraints on the joint action sequence that can be taken by the agents. In this work, we formulate such problems in the framework of constrained cooperative stochastic games. Under this setting, the goal of the agents is to obtain joint action sequence that minimizes a total cost objective criterion subject to total cost penalty/budget functional constraints. To this end, we utilize the Lagrangian formulation and propose actor-critic algorithms. Through experiments on a constrained multi-agent grid world task, we demonstrate that our algorithms converge to near-optimal joint action sequences satisfying the given constraints. © 2019 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.

Item Type:	Conference Paper
Publication:	Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Publisher:	International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Additional Information:	The copyright for this article belongs to International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).
Keywords:	Game theory; Learning algorithms; Machine learning; Multi agent systems; Reinforcement learning; Stochastic systems, Actor-critic algorithm; Function approximation; Functional constraints; Lagrangian formulations; Multi-agent applications; Multi-agent learning; Multi-agent reinforcement learning; Stochastic game, Autonomous agents
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	02 Dec 2022 10:15
Last Modified:	02 Dec 2022 10:15
URI:	https://eprints.iisc.ac.in/id/eprint/78209

Actions (login required)

View Item