ROMIO: A High-Performance, Portable MPI-IO Implementation

Overview

Note: The version of ROMIO described on this page is an old one. We haven't released newer versions of ROMIO as independent packages for a while; they were included as part of MPICH2 and MPICH-1. You can get the latest version of ROMIO when you download MPICH2 or MPICH-1.

ROMIO is a high-performance, portable implementation of MPI-IO, the I/O chapter in MPI-2. Version 1.2.5.1 of ROMIO (January 2003) is freely available. The new features in this version are listed here.

ROMIO runs on at least the following machines: IBM SP; Intel Paragon; HP Exemplar; SGI Origin2000; Cray T3E; NEC SX-4; other symmetric multiprocessors from HP, SGI, DEC, Sun, and IBM; and networks of workstations (Sun, SGI, HP, IBM, DEC, Linux, and FreeBSD). Supported file systems are IBM PIOFS, Intel PFS, HP/Convex HFS, SGI XFS, NEC SFS, PVFS, NFS, and any Unix file system (UFS).

The distribution includes scripts that automate the process of installing ROMIO on any machine.

ROMIO is optimized for noncontiguous access patterns, which are common in parallel applications. It has an optimized implementation of collective I/O, an important optimization in parallel I/O. ROMIO 1.2.5.1 includes everything defined in the MPI-2 I/O chapter except support for file interoperability and user-defined error handlers for files.

C, Fortran, and profiling interfaces are provided for all functions that have been implemented. We have implemented the subarray and distributed array datatype constructors from the MPI-2 miscellaneous chapter, which facilitate I/O involving arrays. We have also implemented the info functions from the MPI-2 misc. chapter, which allow users to pass hints to the implementation.

ROMIO is designed to be used with any MPI implementation. It is, in fact, included as part of several MPI implementations: Version, 1.2.5, is included in MPICH 1.2.5; an earlier version is included in LAM, HP MPI, SGI MPI, and NEC MPI. Version 1.2.5.1 is mainly a bug fix release.

Download ROMIO

ROMIO is freely available and is distributed in the form of source code. Click here to read the Copyright notice.

If you are using MPICH, LAM, HP MPI, SGI MPI, or NEC MPI, you need not download ROMIO separately because ROMIO is already included in those MPI implementations. You may need to get the latest versions of those implementations, however. If you want to use ROMIO with some other MPI implementation, you can download the tar file and follow the instructions for building the library.

We are working towards updating the ROMIO standalone release. If you would like to help test out a pre-release version, we would appreciate any bug reports.

Windows 2000 Version (NTFS)

A version of ROMIO for Windows 2000 is available as part of MPICH2.

Learning MPI-IO

The book Using MPI-2: Advanced Features of the Message-Passing Interface, published by MIT Press, provides a tutorial introduction to all aspects of MPI-2, including parallel I/O. It has lots of example programs.

Using ROMIO on NFS

To use ROMIO on NFS, file locking with fcntl must work correctly on the NFS installation. On some installations, fcntl locks don't work. To get them to work, you need to use Version 3 of NFS, ensure that the lockd daemon is running on all the machines, and have the system administrator mount the NFS file system with the "noac" option (no attribute caching). Turning off attribute caching may reduce performance, but it is necessary for correct behavior. Click here for information on how to set the noac option.

ROMIO and PVFS v2

Support for the PVFS v2 file system has been fairly well-tested by this point. Recent releases of MPICH2 have provided a version of ROMIO with a PVFS v2 ADIO device. Consult this page to see if additional patches might be required. Apply the appropriate patch in the src/mpi/romio directory. More information about ROMIO and PVFS can be found in the ROMIO section of the PVFS Quickstart

Performance Tips

Performance Results

ROMIO Internals

A key component of ROMIO that enables such a portable MPI-IO implementation is an internal abstract I/O device layer called ADIO. Most users of ROMIO will not need to deal with the ADIO layer at all. However, ADIO is useful to those who want to port ROMIO to some other file system. The ROMIO source code and the ADIO paper (ps, pdf, html) will help you get started.

Publications

ROMIO on RCE podcast

In early 2012, Brock Palen and Jeff Squyres chatted with Rajeev Thakur and Rob Latham about ROMIO for the RCE Podcast. That discussion was episode 66.

ROMIO Project Members

Further Information

For further information, send email to mpich-discuss@mcs.anl.gov.