ROMIO: A High-Performance, Portable MPI-IO Implementation
Overview
Note: The version of ROMIO described on this page is an old
one. We haven't released newer versions of ROMIO as independent
packages for a while; they were included as part of MPICH2 and
MPICH-1. You can get the latest version of ROMIO when you download
MPICH2 or MPICH-1.
ROMIO is a high-performance, portable implementation of MPI-IO, the
I/O chapter in MPI-2.
Version 1.2.5.1 of ROMIO (January 2003) is freely available. The new features in this
version are listed here.
ROMIO runs on at least the following machines: IBM SP;
Intel Paragon; HP Exemplar; SGI Origin2000; Cray T3E; NEC SX-4; other
symmetric multiprocessors from HP, SGI, DEC, Sun, and IBM; and networks of
workstations (Sun, SGI, HP, IBM, DEC, Linux, and FreeBSD). Supported
file systems are IBM PIOFS, Intel PFS, HP/Convex HFS, SGI XFS, NEC
SFS, PVFS, NFS, and any
Unix file system (UFS).
The distribution includes scripts that automate the process of
installing ROMIO on any machine.
ROMIO is optimized for noncontiguous access patterns, which are common
in parallel applications. It has an optimized implementation of
collective I/O, an important optimization in parallel
I/O. ROMIO 1.2.5.1 includes everything defined in
the MPI-2 I/O chapter except support for file
interoperability and user-defined error handlers for files.
C, Fortran, and
profiling interfaces are provided for all functions
that have been implemented. We have implemented the subarray and
distributed array datatype constructors from the MPI-2 miscellaneous
chapter, which facilitate I/O involving arrays. We have also implemented
the info functions from the MPI-2 misc. chapter, which allow
users to pass hints to the implementation.
ROMIO is designed to be used with any MPI implementation. It is, in
fact, included as part of several MPI implementations:
Version, 1.2.5, is included in MPICH 1.2.5; an earlier version
is included in LAM, HP MPI, SGI MPI, and NEC MPI.
Version 1.2.5.1 is mainly a bug fix release.
Download ROMIO
ROMIO is freely available and is distributed in the form of source
code. Click here to read the
Copyright notice.
If you are using MPICH, LAM, HP MPI, SGI MPI, or
NEC MPI,
you need not download ROMIO separately because ROMIO is already
included in those MPI implementations. You may need to get the latest
versions of those implementations, however.
If you want to use ROMIO with some other MPI implementation, you can
download the tar file and follow the instructions for building the library.
We are working towards updating the ROMIO standalone release. If you would
like to help test out a pre-release version, we would appreciate any bug
reports.
Windows 2000 Version (NTFS)
A version of ROMIO for Windows 2000 is available as part of
MPICH2.
Learning MPI-IO
The book Using
MPI-2: Advanced Features of the Message-Passing Interface,
published by MIT Press,
provides a tutorial introduction to all aspects of MPI-2, including
parallel I/O. It has lots of example programs.
Using ROMIO on NFS
To use ROMIO on NFS, file locking with fcntl must work correctly on
the NFS installation. On some installations, fcntl locks don't work.
To get them to work, you need to use Version 3 of NFS, ensure that the
lockd daemon is running on all the machines, and
have the system administrator mount the NFS file system with the
"noac" option (no attribute caching). Turning off attribute caching
may reduce performance, but it is necessary for correct behavior.
Click here for information on how to
set the noac option.
ROMIO and PVFS v2
Support for the PVFS v2 file system has been fairly well-tested by this point. Recent releases of MPICH2 have provided a
version of ROMIO with a PVFS v2 ADIO device. Consult this page to see if additional patches might be required. Apply the appropriate patch in the
src/mpi/romio directory. More information about ROMIO and PVFS can
be found in the ROMIO section
of the PVFS Quickstart
Performance Tips
- Specify as much access information as possible in a single function call
by using MPI derived datatypes. For example, if you need to read bytes 0--10,
20--30, and 40--50 from a file, do NOT use three separate read function
calls. Instead, create an appropriate MPI derived datatype
that specifies this layout, define a file view with
MPI_File_set_view, and then use a single read function call.
This will perform MUCH better than with three separate read calls.
- For maximum performance, use the collective I/O functions,
particularly in the case of noncontiguous accesses like the above. See
this paper for further details.
Performance Results
ROMIO Internals
A key component of ROMIO that enables such a portable MPI-IO
implementation is an internal abstract I/O device layer called
ADIO. Most users
of ROMIO will not need to deal with the ADIO layer at all. However,
ADIO is useful to those who want to port ROMIO to some other file
system. The ROMIO source code and the ADIO paper (ps, pdf, html) will help you get started.
ROMIO on RCE podcast
In early 2012, Brock Palen and Jeff Squyres chatted with Rajeev Thakur and Rob
Latham about ROMIO for the RCE Podcast.
That discussion was episode 66.
ROMIO Project Members
Further Information
For further information, send email to mpich-discuss@mcs.anl.gov.