ParNCL and ParGAL: Data Parallel Tools for Post-Processing of Large-Scale Earth Science Data

TitleParNCL and ParGAL: Data Parallel Tools for Post-Processing of Large-Scale Earth Science Data
Publication TypeConference Paper
Year of Publication2013
AuthorsJacob, RL, Krishna, J, Xu, X, Tautges, TJ, Grindeanu, I, Latham, R, Peterson, K, Bochev, P, Haley, M, Brown, D, Brownrigg, R, Shea, D, Huang, W, Middleton, DE
Conference NameInternational Conference on Computational Science (ICCS 2013)
Date Published06/01/2013
PublisherProcedia Computer Science
Conference LocationBarcelona, Spain
Other NumbersANL/MCS-P4041-0313
AbstractEarth science high-performance applications often require extensive analysis of their output in order to complete the scientific goals or produce a visual image or animation. Often this analysis cannot be done in-situ because it requires calculating time-series statistics from state sampled over the entire length of the run or analyzing the relationship between similar time series from previous simulations or observations. Many of the tools used for this post-processing are not themselves high-performance applications but the new Parallel Gridded Analysis Library (ParGAL) provides high-performance data-parallel versions of several common analysis algorithms for data from a structured or unstructured grid simulation. The library builds on several scalable systems including the Mesh Oriented DataBase (MOAB), a library for representing mesh data that supports structured, unstructured finite element and polyhedral grids, The Parallel-NetCDF (PNetCDF) library, and Intrepid, an extensible library for computing operators (such as gradient, curl, divergence, etc.) acting on discretized fields. We have used ParGAL to implement a parallel version of the NCAR Command Language (NCL) a scripting language widely used in the climate community for analysis and visualization. The data-parallel algorithms in ParGAL/ParNCL are both higher performing and more flexible then their serial counterparts.