Argonne National Laboratory

Lessons Learned from Moving Earth System Grid Data Sets Over a 20 Gbps Wide-Area Network

TitleLessons Learned from Moving Earth System Grid Data Sets Over a 20 Gbps Wide-Area Network
Publication TypeJournal Article
Year of Publication2010
AuthorsKettimuthu, R, Sim, A, Gunter, D, Allcock, WE, Bremer, P, Bresnahan, J, Cherry, A, Childers, L, Dart, E, Foster, IT, Harms, K, Hick, J, Lee, J, Link, M, Long, J, Miller, K, Natarajan, V, Pascucci, V, Raffenetti, K, Ressman, D, Williams, D, Wilson, L, Winkler, L
Other NumbersANL/MCS-P1725-0210
AbstractDistributed petascale science computations and experiments require unprecedented wide-area, end-to-end capabilities in the form of high-throughput data transport. The modeling of complex systems, such as climate, at higher fidelity generates proportionately larger volumes of data that are visualized, examined, and studied by widely dispersed scientific research teams to understand climatic imbalances and the potential impacts of future climate change scenarios. In preparation for the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report, the climate community will run the Coupled Model Intercomparison Project phase 5 (CMIP-5) experiments, which are designed to answer crucial questions about future regional climate change and the results of carbon feedback for different mitigation scenarios. The CMIP-5 experiments will generate petabytes of data that must be replicated seamlessly, reliably, and quickly to hundreds of research teams around the globe. As an end-to-end test of the technologies that will be used to perform this task, a multi-disciplinary team of researchers moved a small portion (10 TB) of the multimodel Coupled Model Intercomparison Project, Phase 3 data set used in the IPCC Fourth Assessment Report from three sources—the Argonne Leadership Computing Facility (ALCF), Lawrence Livermore National Laboratory (LLNL) and National Energy Research Scientific Computing Center (NERSC)—to the 2009 Supercomputing conference (SC09) show floor in Portland, Oregon, over circuits provided by DOE’s Energy Sciences Network. This experiment was performed as part of the SC09 Bandwidth Challenge and utilized the system of dedicated data transfer nodes deployed at the ALCF, LLNL, and NERSC. The team achieved a sustained data rate of 15 Gb/s on a 20 Gb/s network. More important, this effort provided critical feedback on how to deploy, tune, and monitor the middleware that will be used to replicate the upcoming petascale climate datasets. In this short paper, we report on obstacles overcome and the key lessons learned from this successful bandwidth challenge effort.