"Toward Simulation-Time Data Analysis and I/O Acceleration on Leadership-Class Systems"
M. Hereld, M. E. Papka, V. Vishwanath
Proc. IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV2011), Providence, RI, , pp. 9-11. Also Preprint ANL/MCS-P1929-0911
Preprint Version: [pdf]
The performance mismatch between computing and I/O components of current-generation HPC systems has made I/O the critical bottleneck for scientific applications. It is therefore critical to make data movement as efficient as possible, and, to facilitate simulation-time data analysis and visualization to reduce the data written to storage. These will be of paramount importance to enabling us to glean novel insights from simulations. We elucidate the challenges faced by applications at extreme scale and present several promising approaches. Next, we will give an overview our work in GLEAN, a flexible framework for data-analysis and I/O at extreme scale that leverages data semantics of applications and fully exploits the diverse system topologies and characteristics. We will discuss the performance of GLEAN for simulation-time analysis and I/O with DOE INCITE and ESP simulations at scale on leadership class systems.