Argonne National Laboratory

Optimization Techniques at the I/O Forwarding Layer

TitleOptimization Techniques at the I/O Forwarding Layer
Publication TypeConference Paper
Year of Publication2010
AuthorsOhta, K, Kimpe, D, Cope, J, Iskra, K, Ross, RB, Ishikawa, Y
Conference NameCluster 2010
Date Published09/2010
Conference LocationHeraklion, Greece

I/O is the critical bottleneck for data-intensive scientific applications on HPC systems and leadership-class machines. Applications running on these systems may encounter bottlenecks
because the I/O systems cannot handle the overwhelming intensity and volume of I/O requests. Applications and systems use I/O forwarding to aggregate and delegate I/O requests to storage systems. In this paper, we present several optimization techniques at the I/O forwarding layer to further reduce I/O bottlenecks on leadership-class computing systems. The first optimization pipelines data transfers so that I/O requests overlap at the network and file system layer. The second optimization merges I/O requests and schedules I/O request delegation to the backend parallel file systems. We implemented these optimizations in the I/O Forwarding Scalability Layer (IOFSL) and evaluated these optimizations on the T2K Open Super Computer at the University of Tokyo and the Surveyor Blue Gene/P system at the Argonne Leadership Computing Facility (ALCF). On both systems, the optimizations improved application I/O throughput and highlighted additional points for I/O contention at the I/O forwarding.