  Markov chains and diffusion processes are indispensable tools in machine
learning and statistics that are used for inference, sampling, and modeling.
With the growth of large-scale datasets, the computational cost associated with
simulating these stochastic processes can be considerable, and many algorithms
have been proposed to approximate the underlying Markov chain or diffusion. A
fundamental question is how the computational savings trade off against the
statistical error incurred due to approximations. This paper develops general
results that address this question. We bound the Wasserstein distance between
the equilibrium distributions of two diffusions as a function of their mixing
rates and the deviation in their drifts. We show that this error bound is tight
in simple Gaussian settings. Our general result on continuous diffusions can be
discretized to provide insights into the computational-statistical trade-off of
Markov chains. As an illustration, we apply our framework to derive
finite-sample error bounds of approximate unadjusted Langevin dynamics. We
characterize computation-constrained settings where, by using fast-to-compute
approximate gradients in the Langevin dynamics, we obtain more accurate samples
compared to using the exact gradients. Finally, as an additional application of
our approach, we quantify the accuracy of approximate zig-zag sampling. Our
theoretical analyses are supported by simulation experiments.
