  Markov chain Monte Carlo methods for exponential family models with
intractable normalizing constant, such as the exchange algorithm, require
simulations of the sufficient statistics at every iteration of the Markov
chain, which often result in expensive computations. Surrogate models for the
likelihood function have been developed to accelerate inference algorithms in
this context. However, these surrogate models tend to be relatively inflexible,
and often provide a poor approximation to the true likelihood function. In this
article, we propose the use of a warped, gradient-enhanced, Gaussian process
surrogate model for the likelihood function, which jointly models the sample
means and variances of the sufficient statistics, and uses warping functions to
capture covariance nonstationarity in the input parameter space. We show that
both the consideration of nonstationarity and the inclusion of gradient
information can be leveraged to obtain a surrogate model that outperforms the
conventional stationary Gaussian process surrogate model when making inference,
particularly in regions where the likelihood function exhibits a phase
transition. We also show that the proposed surrogate model can be used to
improve the effective sample size per unit time when embedded in exact
inferential algorithms. The utility of our approach in speeding up inferential
algorithms is demonstrated on simulated and real-world data.
