ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Adaptive partition migration for irregular graph algorithms on elastic resources

Dindokar, R and Simmhan, Y (2019) Adaptive partition migration for irregular graph algorithms on elastic resources. In: 12th IEEE International Conference on Cloud Computing, CLOUD 2019, 8 - 13 July 2019, Milan, pp. 281-290.

[img] PDF
CLOUD_2019.pdf - Published Version
Restricted to Registered users only

Download (27MB)
Official URL: https://doi.org/10.1109/CLOUD.2019.00-28


Component-centric graph programming models allow distributed graph algorithms to be composed, and executed in an iterative manner on commodity clusters and Clouds. Graphs are partitioned and statically placed on a fixed number of machines before execution. However, many graph algorithms have an irregular execution behavior across partitions in different iterations, which causes resource under-utilization. We propose wo strategies, First Fit Decreasing with Migration Planning (FFDMP) and MinMax, for adaptive partition placement onto an elastic number of Cloud resources for such irregular algorithms. For each iteration, our strategies decide the number of hosts and the placement of partitions on them to balance the compute load, and enact this by migrating partitions between hosts at iteration boundaries. Unlike others, our strategies actively consider the time and cost penalties for moving partitions between hosts, and reduce the overall cost of execution while mitigating any increase in makespan. We implement these strategies on our GoFFish subgraph-centric graph processing platform, and evaluate them for performing Breadth First Search (BFS) on large real-world graphs with 10⁷⁻¹⁰⁹ edges. Our results show that the proposed strategies reduce the median resource cost by 13-38 when compared to a static placement, increase the median makespan by 1-33, which is strictly bound by a given time budget, and also out-perform existing baseline scheduling algorithms from literature. © 2019 IEEE.

Item Type: Conference Paper
Publication: IEEE International Conference on Cloud Computing, CLOUD
Publisher: IEEE Computer Society
Additional Information: The copyright for this article belongs to IEEE Computer Society
Keywords: Budget control; Cloud computing; Computational complexity; Cost reduction; Iterative methods; Scheduling algorithms, Adaptive partitions; Adaptive scaling; Breadth-first search; Commodity clusters; Distributed graph algorithms; Graph algorithms; Graph processing; Migration planning, Clustering algorithms
Department/Centre: Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited: 19 Dec 2022 07:24
Last Modified: 19 Dec 2022 07:24
URI: https://eprints.iisc.ac.in/id/eprint/78507

Actions (login required)

View Item View Item