Argonne National Laboratory

Pattern-driven Parallel I/O Tuning

TitlePattern-driven Parallel I/O Tuning
Publication TypeConference Paper
Year of Publication2015
AuthorsBehzad, B, Prabhat, A, Snir, M
Conference NamePDSW'15
Conference LocationAustin, Texas
Other NumbersANL/MCS-P5340-0415
AbstractThe modern parallel I/O software stack is complex due to the number of configurations for tuning I/O performance. Without proper configuration, I/O becomes a performance bottleneck. As high-performance computing (HPC) is moving towards exascale, poor I/O performance has a significant impact on the runtime of large-scale simulations producing massive amounts of data. In this paper, we focus on developing a framework for tuning parallel I/O configurations automatically. This autotuning framework first traces high-level I/O accesses and analyzes data write patterns. Based on these patterns and historically available tuning parameters for similar patterns, the framework selects best performing configurations. If previous history is unavailable, the framework initiates model-based training to acquire efficient configurations. Our framework includes a runtime system to apply the selected configurations using dynamic linking, without the need for changing code. We test this framework using several I/O kernels extracted from real applications and demonstrate substantial I/O performance benefits.