Using MPI: Portable Parallel Programming with the Message-Passing Interface

by

William Gropp, Ewing Lusk, and Anthony Skjellum

Using MPI, Second Edition

Errata

Table of Contents

1. Background
   • Why Parallel Computing?
   • Obstacles to Progress
   • Why Message Passing?
     • Parallel Computational Models
     • Advantages of the Message-Passing Model
   • Current Message-Passing Systems
   • The MPI Forum
2. What's New about MPI?
   • A New Point of View
   • What's Not New?
   • Basic MPI Concepts
   • Other Interesting Features of MPI
   • Is MPI Large or Small?
     • MPI Is Large (125 Functions)
     • MPI Is Small (6 Functions)
     • MPI Is Whatever Size You Like
   • Decisions Left to the Implementor
3. Using MPI in Simple Programs
   • A First MPI Program
   • Running Your First MPI Program
   • A First MPI Program in C
   • Timing MPI Programs
   • A Self-Scheduling Example: Matrix-Vector Multiplication
   • Studying Parallel Performance
     • Elementary Scalability Calculations
     • Gathering Data on Program Execution
     • Instrumenting a Parallel Program with MPE Logging
     • Events and States
     • Instrumenting the Matrix-Matrix Multiply Program
     • Notes on Implementation of Logging
     • Examining Logfiles with Upshot
   • Using Communicators
   • A Handy Graphics Library for Parallel Programs
   • Application: Determination of Nuclear Structures
   • Summary of a Simple Subset of MPI
4. Intermediate MPI
   • The Poisson Problem
   • Topologies
   • A Code for the Poisson Problem
     • Ordered send and receive
     • Combined send and receive
     • Buffered sends
   • Using Nonblocking Communications
   • Synchronous Sends and ``Safe'' Programs
   • More on Scalability
   • Jacobi with a 2-D Decomposition
   • An MPI Derived Datatype
   • Overlapping Communication and Computation
   • More on Timing Programs
   • Three Dimensions
   • Application: Simulating Vortex Evolution in Superconducting Materials
5. Advanced Message Passing in MPI
   • MPI Datatypes
     • Basic Datatypes and Concepts
     • Derived Datatypes
   • The N-Body Problem
     • Gather
     • Nonblocking Pipeline
     • Moving Particles between Processes
     • Sending Dynamically Allocated Data
     • User-Controlled Data Packing
   • Visualizing the Mandelbrot Set
   • Gaps in Datatypes
6. Parallel Libraries
   • Motivation
     • The Need for Parallel Libraries
     • Common Deficiencies of Message-Passing Systems
     • Review of MPI Features That Support Libraries
   • A First MPI Library
   • Linear Algebra on Grids
     • Mappings and Logical Grids
       • Logical 2-D Grid Topologies
       • Revisiting Grids Using MPI Topology Functions
       • Mapping Functions and Distributions
     • Vectors and Matrices
       • Vectors
       • Matrices
     • Components of a Parallel Library
       • Vector Sum
       • Inner Product
       • Skew Inner product
       • Matrix-Vector Product
   • The LINPACK Benchmark in MPI
   • Strategies for Library Building
7. Other Features of MPI
   • Simulating Shared-Memory Operations
     • Shared vs. Distributed Memory
     • A Counter Example
     • The Shared Counter Using Polling Instead of an Extra Process
     • Shared Memory on Distributed-Memory Machines
   • Application: Full-Configuration Interaction
   • Advanced Collective Operations
     • Data Movement
     • Collective Computation
     • User-Defined Operations
     • Other Collective Operations
   • Intercommunicators
   • Heterogeneous Computing
   • The MPI Profiling Interface
   • Error Handling
     • Error Handlers
     • User-Defined Error Handlers
     • Terminating MPI Programs
   • Environmental Inquiry
     • Processor Name
     • Is MPI Initialized?
   • Other Functions in MPI
   • Application: Computational Fluid Dynamics
     • Parallel Formulation
     • Parallel Implementation
8. Implementing MPI
   • Introduction
   • Sending and Receiving
     • Sending a message
     • Receiving a Message
   • Data Transfer
     • Transfers to and from the ADI
     • Noncontiguous Data
   • Message Queuing
   • Unexpected Messages
   • Device Capabilities and the MPI Library Definition
9. Dusty Decks: Porting Existing Message-Passing Programs to MPI
   • Intel NX
   • IBM EUI
   • TMC CMMD
   • Express
   • PVM 2.4.x
   • PVM 3.2.x
   • p4
   • PARMACS
   • TCGMSG
   • Chameleon
   • Zipcode
   • Where to Learn More
10. Beyond Message Passing
    • Dynamic Processes
    • Threads
    • Action at a Distance
    • Parallel I/O
    • Will There Be an MPI-2?
    • Final Words

A. Summary of MPI Routines and Their Arguments

B. The Model MPI Implementation

C. The MPE Multiprocessing Environment Functions

D. MPI Resources on the Information Superhighway

E. Language Details

A code repository for the examples used in the book is here.

The MPI Standard itself, information on several MPI implementations, and related MPI locations on the Web can be found on the Argonne MPI home page.