  Many fields require computing the trace of the inverse of a large, sparse
matrix. The typical method used for such computations is the Hutchinson method
which is a Monte Carlo (MC) averaging over matrix quadratures. To improve its
convergence, several variance reductions techniques have been proposed. In this
paper, we study the effects of deflating the near null singular value space. We
make two main contributions.
  First, we analyze the variance of the Hutchinson method as a function of the
deflated singular values and vectors. Although this provides good intuition in
general, by assuming additionally that the singular vectors are random unitary
matrices, we arrive at concise formulas for the deflated variance that include
only the variance and mean of the singular values. We make the remarkable
observation that deflation may increase variance for Hermitian matrices but not
for non-Hermitian ones. This is a rare, if not unique, property where
non-Hermitian matrices outperform Hermitian ones. The theory can be used as a
model for predicting the benefits of deflation.
  Second, we use deflation in the context of a large scale application of
"disconnected diagrams" in Lattice QCD. On lattices, Hierarchical Probing (HP)
has previously provided an order of magnitude of variance reduction over MC by
removing "error" from neighboring nodes of increasing distance in the lattice.
Although deflation used directly on MC yields a limited improvement of 30% in
our problem, when combined with HP they reduce variance by a factor of over 60
compared to MC. For this, we pre-computated 1000 smallest singular values of an
ill-conditioned matrix of size 25 million. Using PRIMME and a domain-specific
Algebraic Multigrid preconditioner, we perform one of the largest eigenvalue
computations in Lattice QCD at a fraction of the cost of our trace computation.
