  We consider parallel computation for Gaussian process calculations to
overcome computational and memory constraints on the size of datasets that can
be analyzed. Using a hybrid parallelization approach that uses both threading
(shared memory) and message-passing (distributed memory), we implement the core
linear algebra operations used in spatial statistics and Gaussian process
regression in an R package called bigGP that relies on C and MPI. The approach
divides the matrix into blocks such that the computational load is balanced
across processes while communication between processes is limited. The package
provides an API enabling R programmers to implement Gaussian process-based
methods by using the distributed linear algebra operations without any C or MPI
coding. We illustrate the approach and software by analyzing an astrophysics
dataset with n=67,275 observations.
