New HODLR Software for Gaussian Process Statistical Computations

Researchers in the MACSER project today announced a new release of software for computing Gaussian processes.

Argonne, IL, August 24 2018

Gaussian process models have been used successfully for statistical predictions in the physical sciences, but they are computationally impractical when large datasets are involved.  Instead, approximate methods must be used, typically relying on a covariance matrix that can define characteristics of the Gaussian process such as smoothness and periodicity.

The new software, called KernelMatrices.jl, provides more specialized functionally for Gaussian process computing. Building on recent studies of hierarchical matrices, the software developers extended a specific class of hierarchical matrices known as the hierarchically off-diagonal low-rank (HODLR) matrix structure, and applied hierarchical matrices to the problem of maximum likelihood estimation (MLE) for Gaussian processes.

“We chose the HODLR hierarchical approximation because it enables us to compute the exact derivatives of covariance matrices. Since both the observed and expected information matrices can be computed in quasilinear time, covariance matrices for MLEs can also be estimated efficiently,” said Christopher Geoga, an assistant computational mathematician in the Mathematics and Computer Science (MCS) division at Argonne National Laboratory.

The methodology for the KernelMatrix.jl software suite is described in a companion paper titled Scalable Gaussian Process Computations using Hierarchical Matrices. The paper includes a presentation of the associated mathematics, a demonstration of the scalability of the method, and details of its implementation.

According to the authors, the method offers several advantages.  It is designed to work with kernel matrices where individual elements can be computed efficiently. It provides natural and scalable stochastic, or random, estimators for its gradient and Hessian for a large range of models and offers users many options for numerical optimization.  It avoids many problem-specific computing challenges, such as choosing preconditioners The stochastic gradient and Hessian estimates are sufficiently stable even near the maximum likelihood estimators.
Moreover, the minimizers of the approximated likelihood are nearly identical to the minimizers of the exact likelihood for a wide variety of parameters.

“Putting these together, we present a coherent model that is kernel-independent and provides an attractive and fast way to do exploratory work,” said Mihai Anitescu, a senior computational mathematician in the MCS division and lead of the MACSER center.

KernelMatrix.jl is one of the first software products released by the MACSER center, which is funded by the U.S. Department of Energy to address a grand challenge: how to quantify the occurrence and features of rare, high-impact events in complex energy and environment systems.

The open source software is available for download on the website https://bitbucket.org/cgeoga/kernelmatrices.jl, which includes examples of usage and links to documentation.

The companion paper, Scalable Gaussian Process Computations Using Hierarchical Matrices, by Christopher J. Geoga, Mihai Anitescu, and  Michael L. Stein, is available on the Cornell University Library archive at  https://arxiv.org/abs/1808.03215

Article Entry by Gail Pieper