An Overview of Parallelism Exploitation and Cross-Layer Optimization for Big Data Transfers

TitleAn Overview of Parallelism Exploitation and Cross-Layer Optimization for Big Data Transfers
Publication TypeReport
Year of Publication2013
AuthorsJung, E, Kettimuthu, R
Other NumbersANL/MCS-P5055-1213
AbstractThe data produced by sensors are expected to grow exponentially in the next several years. In particular, e-science applications need to move big data for remote analysis or data distribution, which is required to efficiently utilize distributed resources such as supercomputers, data centers, scientific instruments and the network that connects these facilities. In this article, we evaluate prior work on data transfer over wide-area networks focusing on parallel and cross-layer optimization methods. The cross-layer optimization methods reduce the overhead incurred by many layers or improve the performance of data transfer by appropriately using the information of layers in a holistic way. The goal of this study is to help researchers focus more on unexplored areas, e.g., multipath-aware network protocols, to better handle big data.  
PDFhttp://www.mcs.anl.gov/papers/P5055-1213.pdf