  Kernel models of potential energy surfaces (PES) for polyatomic molecules are
often restricted by a specific choice of the kernel function. This can be
avoided by optimizing the complexity of the kernel function. For regression
problems with very expensive data, the functional form of the model kernels can
be optimized in the Gaussian process (GP) setting through compositional
function search guided by the Bayesian information criterion. However, the
compositional kernel search is computationally demanding and relies on greedy
strategies, which may yield sub-optimal kernels. An alternative strategy of
increasing complexity of GP kernels treats a GP as a Bayesian neural network
(NN) with a variable number of hidden layers, which yields NNGP models. Here,
we present a direct comparison of GP models with composite kernels and NNGP
models for applications aiming at the construction of global PES for polyatomic
molecules. We show that NNGP models of PES can be trained much more efficiently
and yield better generalization accuracy without relying on any specific form
of the kernel function. We illustrate that NNGP models trained by distributions
of energy points at low energies produce accurate predictions of PES at high
energies. We also illustrate that NNGP models can extrapolate in the input
variable space by building the free energy surface of the Heisenberg model
trained in the paramagnetic phase and validated in the ferromagnetic phase. By
construction, composite kernels yield more accurate models than kernels with a
fixed functional form. Therefore, by illustrating that NNGP models outperform
GP models with composite kernels, our work suggests that NNGP models should be
a preferred choice of kernel models for PES.
