  Zigzag and other piecewise deterministic Markov process samplers have
attracted significant interest for their non-reversibility and other appealing
properties for Bayesian posterior computation. Hamiltonian Monte Carlo is
another state-of-the-art sampler, exploiting fictitious momentum to guide
Markov chains through complex target distributions. In this article, we uncover
a remarkable connection between the zigzag sampler and a variant of Hamiltonian
Monte Carlo based on Laplace-distributed momentum. The position and velocity
component of the corresponding Hamiltonian dynamics travels along a zigzag path
paralleling the Markovian zigzag process; however, the dynamics is
non-Markovian in the position-velocity space as the momentum component encodes
non-immediate pasts. This information is partially lost during a momentum
refreshment step, in which we preserve its direction but re-sample magnitude.
In the limit of increasingly frequent momentum refreshments, we prove that
Hamiltonian zigzag converges strongly to its Markovian counterpart. This
theoretical insight suggests that, when retaining full momentum information,
Hamiltonian zigzag can better explore target distributions with highly
correlated parameters by suppressing the diffusive behavior of Markovian
zigzag. We corroborate this intuition by comparing performance of the two
zigzag cousins on high-dimensional truncated multivariate Gaussians, including
a 11,235-dimensional target arising from a Bayesian phylogenetic multivariate
probit model applied to HIV virus data.
