Argonne National Laboratory

Collective I/O Tuning Using Analytical and Machine-Learning Models

TitleCollective I/O Tuning Using Analytical and Machine-Learning Models
Publication TypeConference Paper
Year of Publication2015
AuthorsIsaila, F, Balaprakash, P, Wild, SM, Kimpe, D, Latham, R, Ross, R, Hovland, PD
Conference NameIEEE Cluster 2015
Date Published09/2015
Conference LocationChicago, IL
Other NumbersANL/MCS-P5264-1214
AbstractThe optimization of parallel I/O has become challenging because of the increasing storage hierarchy, performance variability of shared storage systems, and the number of factors in the hardware and software stacks that impact performance. In this paper, we perform an in-depth study of the complexity involved in I/O autotuning and performance modeling, including the architecture, software stack, and noise. We propose a novel hybrid model combining analytical models for communication and storage operations and black-box models for the performance of the individual operations. The experimental results show that the hybrid approach performs significantly better and shows a higher robustness to noise than state-of-the-art machine learning approaches, at the cost of a higher modeling complexity.