  Large scale Gaussian process (GP) regression is infeasible for larger data
sets due to cubic scaling of flops and quadratic storage involved in working
with covariance matrices. Remedies in recent literature focus on
divide-and-conquer, e.g., partitioning into sub-problems and inducing
functional (and thus computational) independence. Such approximations can be
speedy, accurate, and sometimes even more flexible than an ordinary GPs.
However, a big downside is loss of continuity at partition boundaries. Modern
methods like local approximate GPs (LAGPs) imply effectively infinite
partitioning and are thus pathologically good and bad in this regard. Model
averaging, an alternative to divide-and-conquer, can maintain absolute
continuity but often over-smooths, diminishing accuracy. Here we propose
putting LAGP-like methods into a local experts-like framework, blending
partition-based speed with model-averaging continuity, as a flagship example of
what we call precision aggregated local models (PALM). Using $K$ LAGPs, each
selecting $n$ from $N$ total data pairs, we illustrate a scheme that is at most
cubic in $n$, quadratic in $K$, and linear in $N$, drastically reducing
computational and storage demands. Extensive empirical illustration shows how
PALM is at least as accurate as LAGP, can be much faster in terms of speed, and
furnishes continuous predictive surfaces. Finally, we propose sequential
updating scheme which greedily refines a PALM predictor up to a computational
budget.
