Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments

TitleConservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments
Publication TypeReport
Year of Publication2003
AuthorsYang, L, Schopf, JM, Foster, IT
Date Published07/2003
Other NumbersANL/MCS-P1069-0703
Abstract

In heterogeneous and dynamic environments, efficient execution of parallel computations can require mappings of tasks to processors that have both irregular (due to heterogeneity) and time-varying (due to dynamicity) performance. While adaptive domain decomposition techniques have been used to address heterogeneous resource capabilities, temporal variations in those capabilities have seldom been considered. We propose a conservative scheduling policy that uses information about expected future variance in resource capabilities to produce more efficient data mapping decisions. We first present techniques, based on time series predictors that we developed in previous work, for predicting CPU load at some future time point, average CPU load for some future time interval, and variation of CPU load over some future time interval. We then present a family of stochastic scheduling algorithms that exploit such predictions of future availability and variability when making data mapping decisions. Finally, we describe experiments in which we apply our techniques to an astrophysics application. The results of these experiments demonstrate that conservative scheduling can produce execution times that are significantly faster and less variable than other techniques.

PDFhttp://www.mcs.anl.gov/papers/P1069.pdf