  Gaussian process emulators of computationally expensive computer codes
provide fast statistical approximations to model physical processes. The
training of these surrogates depends on the set of design points chosen to run
the simulator. Due to computational cost, such training set is bound to be
limited and quantifying the resulting uncertainty in the hyper-parameters of
the emulator by uni-modal distributions is likely to induce bias. In order to
quantify this uncertainty, this paper proposes a computationally efficient
sampler based on an extension of Asymptotically Independent Markov Sampling, a
recently developed algorithm for Bayesian inference. Structural uncertainty of
the emulator is obtained as a by-product of the Bayesian treatment of the
hyper-parameters. Additionally, the user can choose to perform stochastic
optimisation to sample from a neighbourhood of the Maximum a Posteriori
estimate, even in the presence of multimodality. Model uncertainty is also
acknowledged through numerical stabilisation measures by including a nugget
term in the formulation of the probability model. The efficiency of the
proposed sampler is illustrated in examples where multi-modal distributions are
encountered. For the purpose of reproducibility, further development, and use
in other applications the code used to generate the examples is freely
available for download at https://github.com/agarbuno/paims_codes
