Scheduling Prediction and Monitoring (SPM) Research Group

Who are we

• Faculty
  • Jennifer Schopf
• Graduate Students
  • Christopher Beckmann, Northwestern University
  • Michael Knop, Northwestern University
  • Sudharshan Vazhkudai, Argonne National Lab
  • Xuehai Zhang , University of Chicago
• Summer Students
  • Jeffrey Freschl, UCSC
  • Peter Knepley, Univ Michigan
  • Byoung-dai Lee, University of Minnesota
  • Yu Hu, University of Chicago

What do we do?

• Case based reasoning for replica selection
• Glue-schema
• Grid searcher
• Replica selection using multiple data streams
• WatchTower
• Some of our publications and recent talks

Summer Projects

• One project will involve being able to visualize the architecture of the information infrastructure. More info on the globus monitoring and discovery service (MDS) can be found at http://www.globus.org/mds/. Right now there isn't an easy way to visualize basic data between the components, and we'd like to fix that. This will involve some java programming, and is basically as open-ended as you can imagine - there are a few basic pieces we need, but it can be augmented as interest allows. If there's no interest, a different project for the second half of the summer is possible. As a side-line we'd like to be able to demo some of the basic MDS properties, but we're not really sure the best way to do this, or if that's part of this project or another one. (Starting with something like http://groups.yahoo.com/group/the_gdf/message/3473 is one possibility) Both of these will involve some Java programming, and working with LDAP, although prior ldap experience is not required at all.
• Another project is to construct a demo for how to add in a new information provider - think of this as being able to have a new sensor be part of an information service. There are instructions for how to do this online ( http://www.globus.org/mds/creating_new_providers.pdf) but we'd like to augment this for the tutorials we give into a real demonstration. This will also involve some java programming, and working with ldap, but again, prior ldap experience is not required at all.
• Another project is to update GridSearcher ( http://people.cs.uchicago.edu/~hai/GridSearcher/) a tool that lets you search over the MDS, to handle user-defined schemas. Right now it only deals with the globus defined schemas, but as people add in new sensors we'd like to be able to accommodate them as well. This is a more research-y project, and will involve working with my grad student Xuehai Zhang, and some java programming.
• Another project is more math-oriented, and it will involve being able to do error estimates for predictions, based on some of Sudharshan's prediction work. Papers about his approach are at http://www-unix.mcs.anl.gov/~schopf/Pubs/jmspubs.html, the two with Sudharshan Vazhkudai as lead author. He's generating some good predictions, but we'd also like to estimate ahead of time the errors associated with them, or some kind of confidence interval. We can't use standard statistical techniques because the kinds of distributions involved are really funky. This is a more research-oriented project which will involve working with Sudharshan, and probably using tools like matlab, and a good statistical background is would help.
• We have some continuing work in developing the additional information providers needed for work with the MDS. One project would involve actually working with application scientists and building the needed glue between their sensors and our infrastructure. For example, we'd like to have an information provider that says whether certain services are up, and what there status and load are. This will involve mostly scripting work, and will be fairly straightforward.
• Instance-based Resource Selection
  Christopher Beckmann ( http://www.cs.northwestern.edu/~beckmann ) has been working on the use of instance-based techniques for resource selection. Many distributed computing applications require the use of very large data sets (2GiB), copies of which are available at several sites worldwide. Our work uses records of previous data transfers to select the remote site with the shortest transfer time in the current situation. There are several summer projects available in this area, including applying our selection algorithm to other paradigms and building displays of the selection process. Familiarity with Unix, Perl, and shell scripting is helpful if you're interested in working with the decision algorithm, while experience with Java GUI development is important for the visualization work.
• WatchTower: Panic Button
  This project is a study to determine how users perceive the load state of the Windows NT/2000 machine they are working on. It will involve closeinteractions with Mike Knop ( http://www.cs.northwestern.edu/~knop) and the WatchTower project ( A href ="http://www.cs.northwestern.edu/~knop/research.html"> http://www.cs.northwestern.edu/~knop/wtpb.html
• Extending our earlier regression work between GridFTP transfers and NWS probes, one could consider ping rtts (possibly) as a measure of current networks trends. The argument being, ping rtts are much cheaper than NWS probes and is common. The goal being the forecast of GridFTP throughput. Note in this case that larger ping rtts denote possibly more latency in the network -- this is exact opposite of the NWS case where larger the NWS value, better the network bandwidth. Perhaps a case of negative correlation.
• In the case where we wish to arrive at predictions based on just GridFTP logs between any two sites, we could possibly do some sort of a simulated annealing approach. Just as autoregression is a weighted average solution, we could assign weights based on a particular GridFTP values "staleness" -- staleness increasing as we go back in time. We could then take a subset of transfers, contained by temporal windows, apply a weight to each transfer value based on the staleness-factor and then expose the average as a forecast.