  The space of probability densities is an infinite-dimensional Riemannian
manifold, with Riemannian metrics in two flavors: Wasserstein and Fisher--Rao.
The former is pivotal in optimal mass transport (OMT), whereas the latter
occurs in information geometry---the differential geometric approach to
statistics. The Riemannian structures restrict to the submanifold of
multivariate Gaussian distributions, where they induce Riemannian metrics on
the space of covariance matrices.
  Here we give a systematic description of classical matrix decompositions (or
factorizations) in terms of Riemannian geometry and compatible principal bundle
structures. Both Wasserstein and Fisher--Rao geometries are discussed. The link
to matrices is obtained by considering OMT and information geometry in the
category of linear transformations and multivariate Gaussian distributions.
This way, OMT is directly related to the polar decomposition of matrices,
whereas information geometry is directly related to the $QR$, Cholesky,
spectral, and singular value decompositions. We also give a coherent
description of gradient flow equations for the various decompositions; most
flows are illustrated in numerical examples.
  The paper is a combination of previously known and original results. As a
survey it covers the Riemannian geometry of OMT and polar decompositions
(smooth and linear category), entropy gradient flows, and the Fisher--Rao
metric and its geodesics on the statistical manifold of multivariate Gaussian
distributions. The original contributions include new gradient flows associated
with various matrix decompositions, new geometric interpretations of previously
studied isospectral flows, and a new proof of the polar decomposition of
matrices based an entropy gradient flow.
