An Overview of Parallelism Exploitation and Cross-Layer Optimization for Big Data Transfers

TitleAn Overview of Parallelism Exploitation and Cross-Layer Optimization for Big Data Transfers
Publication TypeConference Paper
Year of Publication2013
AuthorsJung, E, Kettimuthu, R
Conference Name28th IEEE International Parallel & Distributed Processing Symposium
Conference LocationPhoenix, AZ
Other NumbersANL/MCS-P5055-1213
Abstract

The data produced by sensors are expected to grow exponentially in the next several years. In particular, e-science applications need to move big data for remote analysis or data distribution, which is required to efficiently utilize distributed resources such as supercomputers, data centers, scientific instruments and the network that connects these facilities. In this article, we evaluate prior work on data transfer over wide-area networks focusing on parallel and cross-layer optimization methods. The cross-layer optimization methods reduce the overhead incurred by many layers or improve the performance of data transfer by appropriately using the information of layers in a holistic way. The goal of this study is to help researchers focus more on unexplored areas, e.g., multipath-aware network protocols, to better handle big data.

 

PDFhttp://www.mcs.anl.gov/papers/P5055-1213.pdf