An Overview of Parallelism Exploitation and Cross-Layer Optimization for Big Data Transfers
|Title||An Overview of Parallelism Exploitation and Cross-Layer Optimization for Big Data Transfers|
|Publication Type||Conference Paper|
|Year of Publication||2013|
|Authors||Jung, E, Kettimuthu, R|
|Conference Name||28th IEEE International Parallel & Distributed Processing Symposium|
|Conference Location||Phoenix, AZ|
The data produced by sensors are expected to grow exponentially in the next several years. In particular, e-science applications need to move big data for remote analysis or data distribution, which is required to efficiently utilize distributed resources such as supercomputers, data centers, scientific instruments and the network that connects these facilities. In this article, we evaluate prior work on data transfer over wide-area networks focusing on parallel and cross-layer optimization methods. The cross-layer optimization methods reduce the overhead incurred by many layers or improve the performance of data transfer by appropriately using the information of layers in a holistic way. The goal of this study is to help researchers focus more on unexplored areas, e.g., multipath-aware network protocols, to better handle big data.