  It is often the case in Statistics that one needs to compute sums of infinite
series, especially in marginalising over discrete latent variables. This has
become more relevant with the popularization of gradient-based techniques (e.g.
Hamiltonian Monte Carlo) in the Bayesian inference context, for which discrete
latent variables are hard or impossible to deal with. For many commonly used
infinite series, custom algorithms have been developed which exploit specific
features of each problem. General techniques, suitable for a large class of
problems with limited input from the user are less established. We employ basic
results from the theory of infinite series to investigate general,
problem-agnostic algorithms to truncate infinite sums within an arbitrary
tolerance $\varepsilon > 0$ and provide robust computational implementations
with provable guarantees. We compare three tentative solutions to estimating
the infinite sum of interest: (i) a "naive" approach that sums terms until the
terms are below the threshold $\varepsilon$; (ii) a `bounding pair' strategy
based on trapping the true value between two partial sums; and (iii) a `batch'
strategy that computes the partial sums in regular intervals and stops when
their difference is less than $\varepsilon$. We show under which conditions
each strategy guarantees the truncated sum is within the required tolerance and
compare the error achieved by each approach, as well as the number of function
evaluations necessary for each one. A detailed discussion of numerical issues
in practical implementations is also provided. The paper provides some
theoretical discussion of a variety of statistical applications, including raw
and factorial moments and count models with observation error. Finally,
detailed illustrations in the form noisy MCMC for Bayesian inference and
maximum marginal likelihood estimation are presented.
