Toward Optimizing Disk-to-Disk Transfer on 100G Networks

TitleToward Optimizing Disk-to-Disk Transfer on 100G Networks
Publication TypeReport
Year of Publication2013
AuthorsJung, E, Kettimuthu, R, Vishwanath, V
Other NumbersANL/MCS-P4079-0613

The recent emergence of ultra high-speed networks up to 100 Gbps has posed numerous challenges and led to many investigations on efficient protocols to saturate 100 Gbps links. Previous studies showed that RDMA over Converged Ethernet (RoCE) is efficient in terms of CPU load and achievable transfer bandwidth. However, end-to-end data transfers involve many components, not only protocols, affecting overall transfer performance. These components include a disk I/O subsystem, additional computation associated with data streams, and network adapter capacities. For example, achievable bandwidth by RoCE may not be implementable if disk I/O or CPU becomes a bottleneck in end-to-end data transfer.

In this paper, we first model all the system components involved in end-to-end data transfer as a graph. We then formulate the problem whose goal is to achieve maximum data transfer throughput using parallel data flows. Our proposed formulations and solutions are evaluated through experiments on the ESnet 100G testbed. The experimental results show that our approach is several times faster than Globus Online — 8x faster for datasets with many 10MB files and 4x faster for datasets with many 100MB files.