  This paper offers a characterization of fundamental limits on the
classification and reconstruction of high-dimensional signals from
low-dimensional features, in the presence of side information. We consider a
scenario where a decoder has access both to linear features of the signal of
interest and to linear features of the side information signal; while the side
information may be in a compressed form, the objective is recovery or
classification of the primary signal, not the side information. The signal of
interest and the side information are each assumed to have (distinct) latent
discrete labels; conditioned on these two labels, the signal of interest and
side information are drawn from a multivariate Gaussian distribution. With
joint probabilities on the latent labels, the overall signal-(side information)
representation is defined by a Gaussian mixture model. We then provide sharp
sufficient and/or necessary conditions for these quantities to approach zero
when the covariance matrices of the Gaussians are nearly low-rank. These
conditions, which are reminiscent of the well-known Slepian-Wolf and Wyner-Ziv
conditions, are a function of the number of linear features extracted from the
signal of interest, the number of linear features extracted from the side
information signal, and the geometry of these signals and their interplay.
Moreover, on assuming that the signal of interest and the side information obey
such an approximately low-rank model, we derive expansions of the
reconstruction error as a function of the deviation from an exactly low-rank
model; such expansions also allow identification of operational regimes where
the impact of side information on signal reconstruction is most relevant. Our
framework, which offers a principled mechanism to integrate side information in
high-dimensional data problems, is also tested in the context of imaging
applications.
