ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Prediction of Queue Waiting Times for Metascheduling on Parallel Batch Systems

Kumar, Rajath and Vadhiyar, Sathish (2015) Prediction of Queue Waiting Times for Metascheduling on Parallel Batch Systems. In: 18th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), MAY 23, 2014, Phoenix, AZ, pp. 108-128.

[img] PDF
JSSPP_8828_108_2015.pdf - Published Version
Restricted to Registered users only

Download (486kB) | Request a copy
Official URL: http://dx.doi.org/ 10.1007/978-3-319-15789-4_7

Abstract

Prediction of queue waiting times of jobs submitted to production parallel batch systems is important to provide overall estimates to users and can also help meta-schedulers make scheduling decisions. In this work, we have developed a framework for predicting ranges of queue waiting times for jobs by employing multi-class classification of similar jobs in history. Our hierarchical prediction strategy first predicts the point wait time of a job using dynamic k-Nearest Neighbor (kNN) method. It then performs a multi-class classification using Support Vector Machines (SVMs) among all the classes of the jobs. The probabilities given by the SVM for the class predicted using k-NN and its neighboring classes are used to provide a set of ranges of predicted wait times with probabilities. We have used these predictions and probabilities in a meta-scheduling strategy that distributes jobs to different queues/sites in a multi-queue/grid environment for minimizing wait times of the jobs. Experiments with different production supercomputer job traces show that our prediction strategies can give correct predictions for about 77-87% of the jobs, and also result in about 12% improved accuracy when compared to the next best existing method. Experiments with our meta-scheduling strategy using different production and synthetic job traces for various system sizes, partitioning schemes and different workloads, show that the meta-scheduling strategy gives much improved performance when compared to existing scheduling policies by reducing the overall average queue waiting times of the jobs by about 47%.

Item Type: Conference Proceedings
Additional Information: Copy right for this article belongs to the SPRINGER-VERLAG BERLIN, HEIDELBERGER PLATZ 3, D-14197 BERLIN, GERMANY
Department/Centre: Division of Interdisciplinary Research > Supercomputer Education & Research Centre
Depositing User: Id for Latest eprints
Date Deposited: 19 Jul 2015 09:25
Last Modified: 19 Jul 2015 09:25
URI: http://eprints.iisc.ac.in/id/eprint/51836

Actions (login required)

View Item View Item