Toward Simulation-Time Data Analysis and I/O Acceleration on Leadership-Class Systems
|Title||Toward Simulation-Time Data Analysis and I/O Acceleration on Leadership-Class Systems|
|Publication Type||Conference Paper|
|Year of Publication||2011|
|Authors||Hereld, M, Papka, ME, Vishwanath, V|
|Conference Name||Proc. IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV2011)|
|Conference Location||Providence, RI|
The performance mismatch between computing and I/O components of current-generation HPC systems has made I/O the critical bottleneck for scientific applications. It is therefore critical to make data movement as efficient as possible, and, to facilitate simulation-time data analysis and visualization to reduce the data written to storage. These will be of paramount importance to enabling us to glean novel insights from simulations. We elucidate the challenges faced by applications at extreme scale and present several promising approaches. Next, we will give an overview our work in GLEAN, a flexible framework for data-analysis and I/O at extreme scale that leverages data semantics of applications and fully exploits the diverse system topologies and characteristics. We will discuss the performance of GLEAN for simulation-time analysis and I/O with DOE INCITE and ESP simulations at scale on leadership class systems.