Reliable High-Performance Data Transfer via Globus Online

TitleReliable High-Performance Data Transfer via Globus Online
Publication TypeConference Paper
Year of Publication2011
AuthorsFoster, IT, Boverhof, J, Chervenak, AL, Childers, L, DeSchoen, A, Garzoglio, G, Gunter, D, Holzman, B, Kandaswamy, G, Kettimuthu, R, Kordas, J, Livny, M, Martin, S, Mhashilkar, P, Miller, Z, Samak, T, Su, M-H, Tuecke, S, Venkataswamy, V, Ward, C, Weiss, C
Date Published06/2011
Other NumbersANL/MCS-P1904-0611

The SciDAC Center for Enabling Distributed Petascale Science (CEDPS) seeks to accelerate DOE research by eliminating barriers to reliable and performant wide area data movement. A major focus of CEDPS R&D has been the development of Globus Online, a hosted data movement service to which users can hand off responsibility for a range of data movement tasks. Here we introduce the Globus Online system and summarize its impact to date since its launch in November 2010, which include more than 500 terabytes moved, 1000 registered users, and adoption by DOE leadership computing facilities. We also review work aimed at further expanding Globus Onlines utility via: integration with the Condor high-throughput computing system; generalization of the endpoint concept to virtual endpoints with specialized properties such as replication; and performance monitoring as a step towards automated diagnosis of performance problems.