Metascheduling of HPC Jobs in Day-Ahead Electricity Markets

Murali, Prakash and Vadhiyar, Sathish (2018) Metascheduling of HPC Jobs in Day-Ahead Electricity Markets. In: IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 29 (3). pp. 614-627.

PDF IEEE_Tra_Par_Dis_Sys_29-3_614_2018.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

High performance grid computing is a key enabler of large scale collaborative computational science. With the promise of exascale computing, high performance grid systems are expected to incur electricity bills that grow super-linearly over time. In order to achieve cost effectiveness in these systems, it is essential for the scheduling algorithms to exploit electricity price variations, both in space and time, that are prevalent in the dynamic electricity price markets. In this paper, we present a metascheduling algorithm to optimize the placement of jobs in a compute grid which consumes electricity from the day-ahead wholesale market. We formulate the scheduling problem as a Minimum Cost Maximum Flow problem and leverage queue waiting time and electricity price predictions to accurately estimate the cost of job execution at a system. Using trace based simulation with real and synthetic workload traces, and real electricity price data sets, we demonstrate our approach on two currently operational grids, XSEDE and NorduGrid. Our experimental setup collectively constitute more than 433K processors spread across 58 compute systems in 17 geographically distributed locations. Experiments show that our approach simultaneously optimizes the total electricity cost and the average response time of the grid, without being unfair to users of the local batch systems.

Item Type:	Journal Article
Publication:	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
Publisher:	IEEE COMPUTER SOC, 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1314 USA
Additional Information:	Copy right for the article belong to IEEE COMPUTER SOC, 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1314 USA
Department/Centre:	Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre
Date Deposited:	08 Mar 2018 19:09
Last Modified:	25 Aug 2022 08:14
URI:	https://eprints.iisc.ac.in/id/eprint/59106

Actions (login required)

View Item