  The impracticality of posterior sampling has prevented the widespread
adoption of spike-and-slab priors in high-dimensional applications. To
alleviate the computational burden, optimization strategies have been proposed
that quickly find local posterior modes. Trading off uncertainty quantification
for computational speed, these strategies have enabled spike-and-slab
deployments at scales that would be previously unfeasible. We build on one
recent development in this strand of work: the Spike-and-Slab LASSO procedure
of Ro\v{c}kov\'{a} and George (2018). Instead of optimization, however, we
explore multiple avenues for posterior sampling, some traditional and some new.
Intrigued by the speed of Spike-and-Slab LASSO mode detection, we explore the
possibility of sampling from an approximate posterior by performing MAP
optimization on many independently perturbed datasets. To this end, we explore
Bayesian bootstrap ideas and introduce a new class of jittered Spike-and-Slab
LASSO priors with random shrinkage targets. These priors are a key constituent
of the Bayesian Bootstrap Spike-and-Slab LASSO (BB-SSL) method proposed here.
BB-SSL turns fast optimization into approximate posterior sampling. Beyond its
scalability, we show that BB-SSL has a strong theoretical support. Indeed, we
find that the induced pseudo-posteriors contract around the truth at a
near-optimal rate in sparse normal-means and in high-dimensional regression. We
compare our algorithm to the traditional Stochastic Search Variable Selection
(under Laplace priors) as well as many state-of-the-art methods for shrinkage
priors. We show, both in simulations and on real data, that our method fares
superbly in these comparisons, often providing substantial computational gains.
