Case Studies in Storage Access by Loosely Coupled Petascale Applications

TitleCase Studies in Storage Access by Loosely Coupled Petascale Applications
Publication TypeConference Paper
Year of Publication2009
AuthorsWozniak, JM, Wilde, M
Conference NameProceedings of the 4th Annual Workshop on Petascale Data Storage
Date Published10/2009
Conference LocationPortland, Oregon, USA
Other NumbersANL/MCS-P1691-1109

<p>A large number of real-world scientific applications can be characterized as loosely coupled: the communication among tasks is infrequent and can be performed using file operations. While these applications may be ported to large scale machines designed for tightly coupled, massively parallel jobs, direct implementations do not perform well due to the large number of small, latency-bound file accesses. This problem may be overcome through the use of a variety of custom, hand-coded strategies applied at various subsystems of modern near-petascale computers but is a labor intensive process that will become increasingly difficult at the petascale and beyond. This work profiles the essential operations in the I/O workload for five loosely coupled scientific applications. We characterize the I/O workload induced by these applications and offer an analysis to motivate and aid the development of programming tools, I/O subsystems, and filesystems.</p>