Argonne National Laboratory

Preparing PETSc/TAO for Exascale

PETSc is a widely used software library for the scalable solution of linear, nonlinear, and ODE/DAE systems and computation of adjoints (sometimes called sensitivities) of ODE systems. Specifically, we are developing and extending PETSc/TAO in two crucial directions: simulations (that require algebraic solvers and/or time integrators) at the exascale and optimization at the exascale utilizing coupled ensembled simulations.

Under Thrust 1 of our work we are focusing on three topics: partially matrix-free scalable solvers that can efficiently use many-core and GPU-based systems, reduced synchronization algorithms that can scale to larger concurrency than solvers with synchronization points, and performance and data structure optimizations for all the core data structures to better utilize many-core and GPU-based systems as well as provide scalability to the exascale.

Under Thrust 2, we are focusing on libraries for optimization, sensitivity analysis, and uncertainty quantification at the exascale. This work will include implementation of a new, scalable library, called EnsembleLib, for managing a dynamic hierarchical collection of running scalable simulations, where the simulations direction feed results into the optimization, sensitivity analysis, and uncertainty quantification solvers. We will demonstrate the new capabilities provided by EnsembleLib on two exacale subprojects: computation of multiple local minimizers and the associated local sensitivities and stochastic optimization.