Using MPI: Portable Parallel Programming with the Message-Passing Interface

by

William Gropp, Ewing Lusk, and Anthony Skjellum

Information on the first edition of Using MPI is still available.

See our companion book, Using MPI-2, by William Gropp, Ewing Lusk, and Rajeev Thakur, published by MIT Press, 1999; ISBN 0-262-57133-1. The MIT Press page, together with ordering information for this book, can be seen here.

Example Programs

Errata

Porting to MPI from other Message-Passing Systems

Table of Contents

1. Background
   • Why Parallel Computing?
   • Obstacles to Progress
   • Why Message Passing?
     • Parallel Computational Models
     • Advantages of the Message-Passing Model
   • Evolution of Message-Passing Systems
   • The MPI Forum
2. Introduction to MPI
   • Goal
   • What is MPI?
   • Basic MPI Concepts
   • Other Interesting Features of MPI
   • Is MPI Large or Small?
   • Decisions Left to the Implementor
3. Using MPI in Simple Programs
   • A First MPI Program
   • Running Your First MPI Program
   • A First MPI Program in C
   • A First MPI Program in C++
   • Timing MPI Programs
   • A Self-Scheduling Example: Matrix-Vector Multiplication
   • Studying Parallel Performance
     • Elementary Scalability Calculations
     • Gathering Data on Program Execution
     • Instrumenting a Parallel Program with MPE Logging
     • Events and States
     • Instrumenting the Matrix-Matrix Multiply Program
     • Notes on Implementation of Logging
     • Examining Logfiles with Upshot
   • Using Communicators
   • A Handy Graphics Library for Parallel Programs
   • Common Errors and Misunderstandings
   • Application: Quantum Monte Carlo Calculations in Nuclear Physics
   • Summary of a Simple Subset of MPI
4. Intermediate MPI
   • The Poisson Problem
   • Topologies
   • A Code for the Poisson Problem
   • Using Nonblocking Communications
   • Synchronous Sends and ``Safe'' Programs
   • More on Scalability
   • Jacobi with a 2-D Decomposition
   • An MPI Derived Datatype
   • Overlapping Communication and Computation
   • More on Timing Programs
   • Three Dimensions
   • Common Errors and Misunderstandings
   • Application: Simulating Vortex Evolution in Superconducting Materials
5. Advanced Message Passing in MPI
   • MPI Datatypes
     • Basic Datatypes and Concepts
     • Derived Datatypes
     • Understanding Extents
   • The N-Body Problem
     • Gather
     • Nonblocking Pipeline
     • Moving Particles between Processes
     • Sending Dynamically Allocated Data
     • User-Controlled Data Packing
   • Visualizing the Mandelbrot Set
   • Gaps in Datatypes
   • New MPI-2 Datatype Routines
   • More on Datatypes for Structures
   • Deprecated Functions
   • Common Errors and Misunderstandings
6. Parallel Libraries
   • Motivation
     • The Need for Parallel Libraries
     • Common Deficiencies of Message-Passing Systems
     • Review of MPI Features That Support Libraries
   • A First MPI Library
   • Linear Algebra on Grids
     • Mappings and Logical Grids
     • Vectors and Matrices
     • Components of a Parallel Library
   • The LINPACK Benchmark in MPI
   • Strategies for Library Building
   • Examples of Libraries
7. Other Features of MPI
   • Simulating Shared-Memory Operations
     • Shared vs. Distributed Memory
     • A Counter Example
     • The Shared Counter Using Polling Instead of an Extra Process
     • Fairness in Message Passing
     • Exploiting Request-Response Message Patterns
   • Application: Full-Configuration Interaction
   • Advanced Collective Operations
     • Data Movement
     • Collective Computation
     • Common Errors and Misunderstandings
   • Intercommunicators
   • Heterogeneous Computing
   • The MPI Profiling Interface
   • Error Handling
     • Error Handlers
     • An Example of Error Handling
     • User-Defined Error Handlers
     • Terminating MPI Programs
     • MPI-2 Functions for Error Handling
   • The MPI Environment
     • Processor Name
     • Is MPI Initialized?
   • Determining the Version of MPI
   • Other Functions in MPI
   • Application: Computational Fluid Dynamics
     • Parallel Formulation
     • Parallel Implementation
8. Understanding how MPI Implementations Work
   • Introduction
     • Sending Data
     • Receiving Data
     • Rendezvous Protocol
     • Matching Protocols to MPI's Send Modes
     • Performance Implications
     • Alternative MPI Implementation Strategies
     • Tuning MPI Implementations
   • How Difficult is MPI to Implement?
   • Device Capabilities and the MPI Library Definition
   • Reliability of Data Transfer
9. Comparing MPI with Other Systems for Interprocess Communication
   • Sockets
     • Process Startup and Shutdown
     • Handling Faults
   • PVM 3
     • The Basics
     • Miscellaneous Functions
     • Collective Operations
     • MPI Counterparts of Other Functions
     • Features not in MPI
     • Process Startup
     • MPI and PVM related tools
   • Where to Learn More
10. Beyond Message Passing
    • Dynamic Process Management
    • Threads
    • Action at a Distance
    • Parallel I/O
    • MPI-2
    • Will There Be an MPI-3?
    • Final Words

• Summary of MPI-1 Routines and Their Arguments
• The MPICH Implementation of MPI
• The MPE Multiprocessing Environment
• MPI Resources on the World Wide Web
• Language Details

The MPI Standard itself, information on several MPI implementations, and related MPI locations on the Web can be found on the Argonne MPI home page.