ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling

Eshwar, SR and Felipe, LL and Reiffers-Masson, A and Menasche, DS and Thoppe, G (2024) Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling. In: 23rd International Federation for Information Processing on Networking Conference, IFIP Networking 2024, 3 June 2024 through 6 June 2024, Thessaloniki, Greece., pp. 496-501.

[img]
Preview
PDF
IFIP_net_con_2024.pdf - Published Version

Download (788kB) | Preview
Official URL: https://doi.org/10.23919/IFIPNetworking62109.2024....

Abstract

Load balancing and auto scaling are at the core of scalable, contemporary systems, addressing dynamic resource allocation and service rate adjustments in response to workload changes. This paper introduces a novel model and algorithms for tuning load balancers coupled with auto scalers, considering bursty traffic arriving at finite queues. We begin by presenting the problem as a weakly coupled Markov Decision Processes (MDP), solvable via a linear program (LP). However, as the number of control variables of such LP grows combinatorially, we introduce a more tractable relaxed LP formulation, and extend it to tackle the problem of online parameter learning and policy optimization using a two-timescale algorithm based on the LP Lagrangian. Our numerical experiments shed insight into properties of the optimal policy. In particular, we identify a phase transition in the probability of job acceptance as a function of the job dropping costs. The experiments also indicate the efficacy of the proposed online learning method, that learns parameters together with the optimal policy, in converging to the optimal solution of the relaxed LP. In summary, the contributions of this work encompass an analytical model and its LP-based solution approach, together with an online learning algorithm, offering insights into the effective management of distributed systems. © 2024 IFIP.

Item Type: Conference Paper
Publication: 2024 IFIP Networking Conference, IFIP Networking 2024
Publisher: Institute of Electrical and Electronics Engineers Inc.
Additional Information: The copyright for this article belongs to authors.
Keywords: Balancing; Linear programming; Queueing networks; Queueing theory; Resource allocation, Dynamic resource allocations; Linear programs; Linear-programming; Load-Balancing; Markov Decision Processes; Online learning; Optimal policies; Queuing systems; Resources allocation; Scalings, Markov processes
Department/Centre: Division of Electrical Sciences > Computer Science & Automation
Date Deposited: 12 Sep 2024 09:49
Last Modified: 12 Sep 2024 09:49
URI: http://eprints.iisc.ac.in/id/eprint/86141

Actions (login required)

View Item View Item