  This paper describes an approach to simultaneously identify clusters and
estimate cluster-specific regression parameters from the given data. Such an
approach can be useful in learning the relationship between input and output
when the regression parameters for estimating output are different in different
regions of the input space. Variational Inference (VI), a machine learning
approach to obtain posterior probability densities using optimization
techniques, is used to identify clusters of explanatory variables and
regression parameters for each cluster. From these results, one can obtain both
the expected value and the full distribution of predicted output. Other
advantages of the proposed approach include the elegant theoretical solution
and clear interpretability of results. The proposed approach is well-suited for
financial forecasting where markets have different regimes (or clusters) with
different patterns and correlations of market changes in each regime. In
financial applications, knowledge about such clusters can provide useful
insights about portfolio performance and identify the relative importance of
variables in different market regimes. An illustrative example of predicting
one-day S&P change is considered to illustrate the approach and compare the
performance of the proposed approach with standard regression without clusters.
Due to the broad applicability of the problem, its elegant theoretical
solution, and the computational efficiency of the proposed algorithm, the
approach may be useful in a number of areas extending beyond the financial
domain.
