Long-Term Resource Allocation Fairness in Average Markov Decision Process (AMDP) Environment

Ghalme, G and Nair, V and Patil, V and Zhou, Y (2022) Long-Term Resource Allocation Fairness in Average Markov Decision Process (AMDP) Environment. In: 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022, 9 - 13 May 2022, Virtual, Online at Auckland, pp. 525-533.

PDF AAMAS 2022_1_525-533_2022.pdf - Published Version Restricted to Registered users only Download (5MB) | Request a copy

Abstract

Fairness has emerged as an important concern in automated decision-making in recent years, especially when these decisions affect human welfare. In this work, we study fairness in temporally extended decision-making settings, specifically those formulated as Markov Decision Processes (MDPs). Our proposed notion of fairness ensures that each state's long-term visitation frequency is at least a specified fraction. This quota-based notion of fairness is natural in many resource-allocation settings where the dynamics of a single resource being allocated is governed by an MDP and the distribution of the shared resource is captured by its state-visitation frequency. In an average-reward MDP (AMDP) setting, we formulate the problem as a bilinear saddle point program and, for a generative model, solve it using a Stochastic Mirror Descent (SMD) based algorithm. The proposed solution guarantees a simultaneous approximation on the expected average-reward and fairness requirement. We give sample complexity bounds for the proposed algorithm and validate our theoretical results with experiments on simulated data.

Item Type:	Conference Paper
Publication:	Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Publisher:	International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Additional Information:	The copyright for this article belongs to the International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).
Keywords:	Autonomous agents; Behavioral research; Decision making; Learning algorithms; Markov processes; Multi agent systems; Resource allocation; Stochastic models; Stochastic systems, Automated decision making; Average reward; Decisions makings; Fairness; Human welfare; Markov Decision Processes; Process environment; Reinforcement learnings; Resources allocation; Shared resources, Reinforcement learning
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	04 Aug 2022 11:29
Last Modified:	04 Aug 2022 11:29
URI:	https://eprints.iisc.ac.in/id/eprint/75340

Actions (login required)

View Item