Prediction of Queue Waiting Times for Metascheduling on Parallel Batch Systems

Kumar, Rajath and Vadhiyar, Sathish (2015) Prediction of Queue Waiting Times for Metascheduling on Parallel Batch Systems. In: 18th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), MAY 23, 2014, Phoenix, AZ, pp. 108-128.

PDF JSSPP_8828_108_2015.pdf - Published Version Restricted to Registered users only Download (486kB) | Request a copy

Abstract

Prediction of queue waiting times of jobs submitted to production parallel batch systems is important to provide overall estimates to users and can also help meta-schedulers make scheduling decisions. In this work, we have developed a framework for predicting ranges of queue waiting times for jobs by employing multi-class classification of similar jobs in history. Our hierarchical prediction strategy first predicts the point wait time of a job using dynamic k-Nearest Neighbor (kNN) method. It then performs a multi-class classification using Support Vector Machines (SVMs) among all the classes of the jobs. The probabilities given by the SVM for the class predicted using k-NN and its neighboring classes are used to provide a set of ranges of predicted wait times with probabilities. We have used these predictions and probabilities in a meta-scheduling strategy that distributes jobs to different queues/sites in a multi-queue/grid environment for minimizing wait times of the jobs. Experiments with different production supercomputer job traces show that our prediction strategies can give correct predictions for about 77-87% of the jobs, and also result in about 12% improved accuracy when compared to the next best existing method. Experiments with our meta-scheduling strategy using different production and synthetic job traces for various system sizes, partitioning schemes and different workloads, show that the meta-scheduling strategy gives much improved performance when compared to existing scheduling policies by reducing the overall average queue waiting times of the jobs by about 47%.

Item Type:	Conference Proceedings
Series.:	Lecture Notes in Computer Science
Publisher:	SPRINGER-VERLAG BERLIN
Additional Information:	Copy right for this article belongs to the SPRINGER-VERLAG BERLIN, HEIDELBERGER PLATZ 3, D-14197 BERLIN, GERMANY
Department/Centre:	Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre
Date Deposited:	19 Jul 2015 09:25
Last Modified:	19 Jul 2015 09:25
URI:	http://eprints.iisc.ac.in/id/eprint/51836

Actions (login required)

View Item