AutoBoT: Resilient and Cost-Effective Scheduling of a Bag of Tasks on Spot VMs

Varshney, Prateeksha and Simmhan, Yogesh (2019) AutoBoT: Resilient and Cost-Effective Scheduling of a Bag of Tasks on Spot VMs. In: IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 30 (7). pp. 1512-1527.

PDF iee_tra_par_dis_sys_30-7_1512_2019.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

Many data and task parallel applications can be modeled as a Bag of Tasks (BoT), and scheduled on distributed systems such as Grids, Clusters, and Clouds. We propose AutoBoT, a collection of scheduling strategies for BoTs with hard deadlines on Cloud Virtual Machines (VMs), to lower the overall monetary cost-a distinctive factor for Clouds. Besides reliable fixed-price VMs, AutoBoT uniquely reduces costs by including preemptible spot-priced VMs that are much cheaper, but are unreliable and have time-variant pricing. It guarantees timely completion by making active runtime decisions on pricing, number of VMs to acquire/release, and on task placement, checkpointing and migration. Our rigorous simulations of 7 Million BoT runs sampled from the Google cluster workload uses a realistic Cloud model and 6 months of Amazon EC2 pricing data to compare AutoBoT against two baseline algorithms. We analyze the impact of BoT size, data centers, time periods, deadline duration, loss budget and checkpointing strategies. AutoBoT often gives approximate to 80% profit and rare but bounded losses, compared to using only fixed-price VMs. Further, its 100 percent completion guarantee is 23-42 percent better than using only spot-priced VMs which offer a similar profit.

Item Type:	Journal Article
Publication:	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
Publisher:	IEEE COMPUTER SOC
Additional Information:	copyright for this article belongs to IEEE COMPUTER SOC
Keywords:	Scheduling; cloud computing; bag of tasks; heuristics; checkpointing; reliability; spot pricing
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	05 Aug 2019 05:34
Last Modified:	05 Aug 2019 05:34
URI:	http://eprints.iisc.ac.in/id/eprint/63173

Actions (login required)

View Item