  Sampling from the joint posterior distribution of Gaussian mixture models
(GMMs) via standard Markov chain Monte Carlo (MCMC) imposes several
computational challenges, which have prevented a broader full Bayesian
implementation of these models. A growing body of literature has introduced the
Weighted Likelihood Bootstrap and the Weighted Bayesian Bootstrap as
alternatives to MCMC sampling. The core idea of these methods is to repeatedly
compute maximum a posteriori (MAP) estimates on many randomly weighted
posterior densities. These MAP estimates then can be treated as approximate
posterior draws. Nonetheless, a central question remains unanswered: How to
select the distribution of the random weights under arbitrary sample sizes.
Thus, we introduce the Bayesian Optimized Bootstrap (BOB), a computational
method to automatically select the weights distribution by minimizing, through
Bayesian Optimization, a black-box and noisy version of the reverse KL
divergence between the Bayesian posterior and an approximate posterior obtained
via random weighting. Our proposed method allows for uncertainty
quantification, approximate posterior sampling, and embraces recent
developments in parallel computing. We show that BOB outperforms competing
approaches in recovering the Bayesian posterior, while retaining key
theoretical properties from existing methods. BOB's performance is demonstrated
through extensive simulations, along with real-world data analyses.
