  We describe and analyze a variance reduction approach for Monte Carlo (MC)
sampling that accelerates the estimation of statistics of computationally
expensive simulation models using an ensemble of models with lower cost. These
lower cost models --- which are typically lower fidelity with unknown
statistics --- are used to reduce the variance in statistical estimators
relative to a MC estimator with equivalent cost. We derive the conditions under
which our proposed approximate control variate framework recovers existing
multi-model variance reduction schemes as special cases. We demonstrate that
these existing strategies use recursive sampling strategies, and as a result,
their maximum possible variance reduction is limited to that of a control
variate algorithm that uses only a single low-fidelity model with known mean.
This theoretical result holds regardless of the number of low-fidelity models
and/or samples used to build the estimator. We then derive new sampling
strategies within our framework that circumvent this limitation to make
efficient use of all available information sources. In particular, we
demonstrate that a significant gap can exist, of orders of magnitude in some
cases, between the variance reduction achievable by using a single low-fidelity
model and our non-recursive approach. We also present initial sample allocation
approaches for exploiting this gap. They yield the greatest benefit when
augmenting the high-fidelity model evaluations is impractical because, for
instance, they arise from a legacy database. Several analytic examples and an
example with a hyperbolic PDE describing elastic wave propagation in
heterogeneous media are used to illustrate the main features of the
methodology.
