  Riemannian manifold Hamiltonian Monte Carlo (RMHMC) is a sampling algorithm
that seeks to adapt proposals to the local geometry of the posterior
distribution. The specific form of the Hamiltonian used in RMHMC necessitates
{\it implicitly-defined} numerical integrators in order to sustain
reversibility and volume-preservation, two properties that are necessary to
establish detailed balance of RMHMC. In practice, these implicit equations are
solved to a non-zero convergence tolerance via fixed-point iteration. However,
the effect of these convergence thresholds on the ergodicity and computational
efficiency properties of RMHMC are not well understood. The purpose of this
research is to elucidate these relationships through numerous case studies. Our
analysis reveals circumstances wherein the RMHMC algorithm is sensitive, and
insensitive, to these convergence tolerances. Our empirical analysis examines
several aspects of the computation: (i) we examine the ergodicity of the RMHMC
Markov chain by employing statistical methods for comparing probability
measures based on collections of samples; (ii) we investigate the degree to
which detailed balance is violated by measuring errors in reversibility and
volume-preservation; (iii) we assess the efficiency of the RMHMC Markov chain
in terms of time-normalized ESS. In each of these cases, we investigate the
sensitivity of these metrics to the convergence threshold and further
contextualize our results in terms of comparison against Euclidean HMC. We
propose a method by which one may select the convergence tolerance within a
Bayesian inference application using techniques of stochastic approximation and
we examine Newton's method, an alternative to fixed point iterations, which can
eliminate much of the sensitivity of RMHMC to the convergence threshold.
