  The paper proposes a Riemannian Manifold Hamiltonian Monte Carlo sampler to
resolve the shortcomings of existing Monte Carlo algorithms when sampling from
target densities that may be high dimensional and exhibit strong correlations.
The method provides a fully automated adaptation mechanism that circumvents the
costly pilot runs required to tune proposal densities for Metropolis-Hastings
or indeed Hybrid Monte Carlo and Metropolis Adjusted Langevin Algorithms. This
allows for highly efficient sampling even in very high dimensions where
different scalings may be required for the transient and stationary phases of
the Markov chain. The proposed method exploits the Riemannian structure of the
parameter space of statistical models and thus automatically adapts to the
local manifold structure at each step based on the metric tensor. A
semi-explicit second order symplectic integrator for non-separable Hamiltonians
is derived for simulating paths across this manifold which provides highly
efficient convergence and exploration of the target density. The performance of
the Riemannian Manifold Hamiltonian Monte Carlo method is assessed by
performing posterior inference on logistic regression models, log-Gaussian Cox
point processes, stochastic volatility models, and Bayesian estimation of
parameter posteriors of dynamical systems described by nonlinear differential
equations. Substantial improvements in the time normalised Effective Sample
Size are reported when compared to alternative sampling approaches. Matlab code
at \url{http://www.dcs.gla.ac.uk/inference/rmhmc} allows replication of all
results.
