Stochastic Approximation of Score Functions for Gaussian Processes

TitleStochastic Approximation of Score Functions for Gaussian Processes
Publication TypeJournal Article
Year of Publication2013
AuthorsStein, ML, Chen, J, Anitescu, M
JournalAnnals of Applied Statistics
Date Published06/2012
Other NumbersANL/MCS-P2091-0512

We discuss the statistical properties of a recently introduced unbiased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves O(n) storage and nearly O(n) computational effort per optimization step, where n is the number of data sites. Here, we prove that if the condition number of the covariance matrix is bounded, then the approximate score equations are nearly optimal in a well-defined sense. Therefore not only is the approximation efficient to compute, but it also has comparable statistical properties to the exact maximum likelihood estimates. We discuss a modification of the stochastic approximation in which design elements of the stochastic terms mimic patterns from a 2n factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can produce a substantial improvement over random designs. Our findings are validated by numerical experiments on up to 1 million data sites that include fitting of numerical output from a problem in geodynamics.