Argonne National Laboratory

Performance Evaluation of Darshan 3.0.0 on the Cray XC30

TitlePerformance Evaluation of Darshan 3.0.0 on the Cray XC30
Publication TypeReport
Year of Publication2016
AuthorsSnyder, S, Carns, PH, Harms, K, Latham, R, Ross, R
Series TitleANL/MCS-Tm-362
Date Published04/2016
InstitutionArgonne National Laboratory
Report NumberANL/MCS-TM-362
AbstractDarshan is a lightweight I/O characterization tool used to gather and summarize salient I/O workload statistics from HPC applications. Darshan was designed to minimize any possible perturbations of an application’s performance, leading it to be enabled by default on a number of production HPC systems. For each file accessed by a given application, Darshan records the count and types of I/O operations performed, histograms of access sizes, cumulative timers on the amount of time spent doing I/O, and other statistical data. This type of data has proved invaluable in understanding and improving the I/O performance of HPC applications. Darshan 3.0.0 is the new modularized version of the traditional Darshan library and file format, allowing users to easily add more in-depth I/O characterization data to Darshan logs. In this work we perform an empirical evaluation of Darshan 3.0.0 to ensure that it continues to meet performance expectations for broad deployment. In particular, we evaluate the imposed overhead on instrumented I/O operations, time taken to shut down and gener- ate corresponding Darshan log files, and resultant log file sizes for different workloads. These performance results are compared to results of Darshan 2.3.0 on the Edison XC30 system at NERSC to determine whether the new version is lightweight enough to run full-time on production HPC systems. Our evaluation shows that Darshan has limited impact on application I/O performance and can fully generate a corresponding log file for most application workloads in under two seconds.