Production I/O Characterization on the Cray XE6

TitleProduction I/O Characterization on the Cray XE6
Publication TypeConference Paper
Year of Publication2013
AuthorsCarns, PH, Yao, Y, Harms, K, Latham, R, Ross, RB, Antypas, K
Conference NameCUG2013
Date PublishedMay 9, 2013
Conference LocationNapa Valley, CA
Other NumbersANL/MCS-P4057-0413

I/O performance is an increasingly important factor in the productivity of large-scale high-performance computing systems such as Hopper, a 153,216-core Cray XE6 system operated by the National Energy Research Scientific Computing Center. The scientific workload diversity of such systems presents a challenge for I/O performance tuning, however. Applications vary in terms of data volume, I/O strategy, and access method, making it difficult to consistently evaluate and enhance their I/O performance.

We have adapted the Darshan I/O characterization tool for use on Hopper in order to address this challenge. Darshan is an I/O instrumentation library that collects I/O access pattern information from large-scale production applications with minimal overhead. In this paper we present our experiences in deploying Darshan on the Cray XE6 platform, including performance evaluation and a case study of how to identify applications that can benefit most from I/O performance tuning. Darshan was automatically enabled for all Hopper users on December 2012 and instruments over 5,000 jobs per day as of April 2013.