  In latent variable models the parameter estimation can be implemented by
using the joint or the marginal likelihood, based on independence or
conditional independence assumptions. The same dilemma occurs within the
Bayesian framework with respect to the estimation of the Bayesian marginal (or
integrated) likelihood, which is the main tool for model comparison and
averaging. In most cases, the Bayesian marginal likelihood is a high
dimensional integral that cannot be computed analytically and a plethora of
methods based on Monte Carlo integration (MCI) are used for its estimation. In
this work, it is shown that the joint MCI approach makes subtle use of the
properties of the adopted model, leading to increased error and bias in finite
settings. The sources and the components of the error associated with
estimators under the two approaches are identified here and provided in exact
forms. Additionally, the effect of the sample covariation on the Monte Carlo
estimators is examined. In particular, even under independence assumptions the
sample covariance will be close to (but not exactly) zero which surprisingly
has a severe effect on the estimated values and their variability. To address
this problem, an index of the sample's divergence from independence is
introduced as a multivariate extension of covariance. The implications
addressed here are important in the majority of practical problems appearing in
Bayesian inference of multi-parameter models with analogous structures.
