Designing and Building Parallel Programs:
An Introduction to Parallel Programming

A Full-Day Tutorial on Parallel Programming, presented at:

Ian Foster, Bill Gropp, Carl Kesselman, Charles Koelbel. (We sometimes substitute one or more of the speakers.)

In this tutorial, we provide a comprehensive introduction to the techniques and tools used to write parallel programs. Our goal is to communicate the practical information required by scientists, engineers, and educators who need to write parallel programs or to teach parallel programming. First, we introduce principles of parallel program design, touching upon relevant topics in architecture, algorithms, and performance modeling. Then, we describe the parallel programming standards High Performance Fortran and Message Passing Interface, and two modern parallel languages, Compositional C++ and Fortran M.

The tutorial is based on the textbook Designing and Building Parallel Programs (Addison-Wesley, 1995), which covers the same material in considerable detail. A novel feature of this book is that it is also available in HTML format on the World Wide Web at URL http://www.mcs.anl.gov/dbpp/. The Web version provides a convenient hypertext version of the book, plus links to the programming tools described in this tutorial and numerous other online resources.


Course Materials

We typically provide a copy of Designing and Building Parallel Programs to each attendee. This textbook provides a comprehensive, 400-page treatment of the tutorial material that attendees can take home with them, and we believe will add greatly to the value and appeal of the tutorial. Copies of the 220 tutorial overheads.


Level

This tutorial is intended to be primarily introductory in nature, in that it assumes no prior knowledge of parallel programming. As it will also cover modern parallel languages and libraries that may not be familiar to many, even those active in parallel programming, we would categorize it as 60% introductory, 40% intermediate.


Structure

The tutorial is divided into six parts, each of about one hour in length. These sessions, and their presenters, are as follows:


Brief Description

This tutorial is intended as a highly practical introduction to parallel programming. It comprises two main parts. In the first two hours, we provide an introduction to parallel program design, touching upon parallel computer architecture, parallel algorithms, and performance analysis. This material corresponds to Chapters 1-4 of Designing and Building Parallel Programs.

In the remaining four hours, we provide a practitioner's guide to four parallel programming tools: the languages CC++ and Fortran M, and the de facto standards High Performance Fortran and Message Passing Interface. This material corresponds to Chapters 5-8 of Designing and Building Parallel Programs. In our discussion of each of these tools, we are careful to relate language and library features to the parallel program design techniques introduced in the first part of the tutorial. We also use the same examples when this is appropriate.

By the end of the tutorial, attendees have been exposed to the essential information required to develop parallel programs, and (by virtue of the textbook to be distributed with the course notes) know where to turn for additional information, example programs, programming exercises, and parallel programming tools.

The material presented in this tutorial should be of interest to three groups of people:

We believe that this tutorial complements rather than competes with more focused tutorials that describe only a single system (e.g., HPF or MPI). Some tutorial attendees will be looking for a detailed discussion of HPF or MPI; others will be seeking a more general introduction to parallel programming. We expect these groups of people to be relatively disjoint.


Visual Materials

The tutorial overheads have been prepared using Powerpoint, but at this stage only some have been converted into a form suitable for inclusion in an HTML document. All will be available via the Web eventually. Here are two examples (select the images to see larger versions):

. .

Some of the slides used by Dr Koelbel in introducing HPF are also accessible via the Web.

We make use of the many figures and example programs included in Designing and Building Parallel Programs. These figures and programs are available online, for convenient access by educators.


Vitae

All four presenters are experts in their field, and have many years experience in parallel programming research and development. All have presented tutorials of this sort on numerous occasions.

Dr. Ian Foster is a Scientist in the Mathematics and Computer Science Division at Argonne National Laboratory and a participant in the Center for Research on Parallel Computation. He is the author of Designing and Building Parallel Programs and the co-developer of Fortran M. A short vita is available for Dr Foster. (Vitae for the other lecturers can be provided on request.)

Dr. Bill Gropp is a Scientist in the Mathematics and Computer Science Division at Argonne National Laboratory. He has published extensively in parallel computing, and was one of the most active members of the MPI Forum that designed the Message Passing Interface. He is the author, with Rusty Lusk and Tony Skjellum, of the book Using MPI.

Dr. Carl Kesselman is a Member of the Beckman Institute at the California Institute of Technology. He is the codesigner with Mani Chandy of Compositional C++, and has developed a portable compiler for this language. He is an active participant in the Center for Research on Parallel Computation.

Dr. Charles Koelbel is a Scientist in the Department of Computer Science at Rice University. He was the Executive Director of the HPF Forum that designed High Performance Forum and is one of the authors of The High Performance Fortran Handbook. He is also an active participant in the Center for Research on Parallel Computation.


Designing and Building Parallel Programs / [email protected]