  Variable selection, or more generally, model reduction is an important aspect
of the statistical workflow aiming to provide insights from data. In this
paper, we discuss and demonstrate the benefits of using a reference model in
variable selection. A reference model acts as a noise-filter on the target
variable by modeling its data generating mechanism. As a result, using the
reference model predictions in the model selection procedure reduces the
variability and improves stability leading to improved model selection
performance. Assuming that a Bayesian reference model describes the true
distribution of future data well, the theoretically preferred usage of the
reference model is to project its predictive distribution to a reduced model
leading to projection predictive variable selection approach. Alternatively,
reference models may also be used in an ad-hoc manner in combination with
common variable selection methods. In several numerical experiments, we
investigate the performance of the projective prediction approach as well as
alternative variable selection methods with and without reference models. Our
results indicate that the use of reference models generally translates into
better and more stable variable selection. Additionally, we demonstrate that
the projection predictive approach shows superior performance as compared to
alternative variable selection methods independently of whether or not they use
reference models.
