Kumar, Rajath and Vadhiyar, Sathish (2015) Prediction of Queue Waiting Times for Metascheduling on Parallel Batch Systems. In: 18th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), MAY 23, 2014, Phoenix, AZ, pp. 108-128.
PDF
JSSPP_8828_108_2015.pdf - Published Version Restricted to Registered users only Download (486kB) | Request a copy |
Abstract
Prediction of queue waiting times of jobs submitted to production parallel batch systems is important to provide overall estimates to users and can also help meta-schedulers make scheduling decisions. In this work, we have developed a framework for predicting ranges of queue waiting times for jobs by employing multi-class classification of similar jobs in history. Our hierarchical prediction strategy first predicts the point wait time of a job using dynamic k-Nearest Neighbor (kNN) method. It then performs a multi-class classification using Support Vector Machines (SVMs) among all the classes of the jobs. The probabilities given by the SVM for the class predicted using k-NN and its neighboring classes are used to provide a set of ranges of predicted wait times with probabilities. We have used these predictions and probabilities in a meta-scheduling strategy that distributes jobs to different queues/sites in a multi-queue/grid environment for minimizing wait times of the jobs. Experiments with different production supercomputer job traces show that our prediction strategies can give correct predictions for about 77-87% of the jobs, and also result in about 12% improved accuracy when compared to the next best existing method. Experiments with our meta-scheduling strategy using different production and synthetic job traces for various system sizes, partitioning schemes and different workloads, show that the meta-scheduling strategy gives much improved performance when compared to existing scheduling policies by reducing the overall average queue waiting times of the jobs by about 47%.
Item Type: | Conference Proceedings |
---|---|
Series.: | Lecture Notes in Computer Science |
Publisher: | SPRINGER-VERLAG BERLIN |
Additional Information: | Copy right for this article belongs to the SPRINGER-VERLAG BERLIN, HEIDELBERGER PLATZ 3, D-14197 BERLIN, GERMANY |
Department/Centre: | Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre |
Date Deposited: | 19 Jul 2015 09:25 |
Last Modified: | 19 Jul 2015 09:25 |
URI: | http://eprints.iisc.ac.in/id/eprint/51836 |
Actions (login required)
View Item |