The message-passing model of parallel computation has emerged as an expressive, efficient, and well-understood paradigm for parallel programming. Until recently, the syntax and precise semantics of each message-passing library implementation were different from the others, although many of the general semantics were similar. The proliferation of message-passing library designs from both vendors and users was appropriate for a while, but eventually it was seen that enough consensus on requirements and general semantics for message-passing had been reached that an attempt at standardization might usefully be undertaken.
The process of creating a standard to enable portability of message-passing applications codes began at a workshop on Message Passing Standardization in April 1992, and the Message Passing Interface (MPI) Forum organized itself at the Supercomputing '92 Conference. During the next eighteen months the MPI Forum met regularly, and Version 1.0 of the MPI Standard was completed in May 1994 [16,36]. Some clarifications and refinements were made in the spring of 1995, and Version 1.1 of the MPI Standard is now available [17]. For a detailed presentation of the Standard itself, see [42]; for a tutorial approach to MPI, see [29]. In this paper we assume that the reader is relatively familiar with the MPI specification, but we provide a brief overview in Section Brief Overview of MPI .
The project to provide a portable implementation of MPI began at the same time as the MPI definition process itself. The idea was to provide early feedback on decisions being made by the MPI Forum and provide an early implementation to allow users to experiment with the definitions even as they were being developed. Targets for the implementation were to include all systems capable of supporting the message-passing model. MPICH is a freely available, complete implementation of the MPI specification, designed to be both portable and efficient. The ``CH'' in MPICH stands for ``Chameleon,'' symbol of adaptability to one's environment and thus of portability. Chameleons are fast, and from the beginning a secondary goal was to give up as little efficiency as possible for the portability.
MPICH is thus both a research project and a software development project. As a research project, its goal is to explore methods for narrowing the gap between the programmer of a parallel computer and the performance deliverable by its hardware. In MPICH, we adopt the constraint that the programming interface will be MPI, reject constraints on the architecture of the target machine, and retain high performance (measured in terms of bandwidth and latency for message-passing operations) as a goal. As a software project, MPICH's goal is to promote the adoption of the MPI Standard by providing users with a free, high-performance implementation on a diversity of platforms, while aiding vendors in providing their own customized implementations. The extent to which these goals have been achieved is the main thrust of this paper.
The rest of this paper is organized as follows. Section Background gives a short overview of MPI and briefly describes the precursor systems that influenced MPICH and enabled it to come into existence so quickly. In Section Portability and Performance we document the extent of MPICH's portability and present results of a number of performance measurements. In Section Architecture of MPICH we describe in some detail the software architecture of MPICH, which comprises the results of our research in combining portability and performance. In Section Selected Subsystems we present several specific aspects of the implementation that merit more detailed analysis. Section Toward a Portable Parallel Programming Environment describes a family of supporting programs that surround the core MPI implementation and turn MPICH into a portable environment for developing parallel applications. In Section Software Management Techniques and Tools we describe how we as a small, distributed group have combined a number of freely available tools in the Unix environment to enable us to develop, distribute, and maintain MPICH with a minimum of resources. In the course of developing MPICH, we have learned a number of lessons from the challenges posed (both accidentally and deliberately) for MPI implementors by the MPI specification; these lessons are discussed in Section Lessons Learned . Finally, Section Status and Plans describes the current status of MPICH (Version 1.0.12 as of February 1996) and outlines our plans for future development.