The Impact of File Systems on MPI-IO Scalability

TitleThe Impact of File Systems on MPI-IO Scalability
Publication TypeReport
Year of Publication2004
AuthorsLatham, R, Ross, RB, Thakur, R
Date Published06/2004
Other NumbersANL/MCS-P1182-0604

As the number of nodes in cluster systems continues to grow, leveraging scalable algorithms in all aspects of such systems becomes key to maintaining performance. While scalable algorithms have been applied successfully in some areas of parallel I/O, many operations are still performed in an uncoordinated manner. In this work we consider, in three file system scenarios, the possibilities for applying scalable algorithms to the many operations that make up the MPI-IO interface. From this evaluation we extract a set of file system characteristics that aid in the implementation of scalable MPI-IO implementations.