  In statistics, independent, identically distributed random samples do not
carry a natural ordering, and their statistics are typically invariant with
respect to permutations of their order. Thus, an $n$-sample in a space $M$ can
be considered as an element of the quotient space of $M^n$ modulo the
permutation group. The present paper takes this definition of sample space and
the related concept of orbit types as a starting point for developing a
geometric perspective on statistics. We aim at deriving a general mathematical
setting for studying the behavior of empirical and population means in spaces
ranging from smooth Riemannian manifolds to general stratified spaces. We fully
describe the orbifold and path-metric structure of the sample space when $M$ is
a manifold or path-metric space, respectively. These results are non-trivial
even when $M$ is Euclidean. We show that the infinite sample space exists in a
Gromov-Hausdorff type sense and coincides with the Wasserstein space of
probability distributions on $M$. We exhibit Fr\'echet means and $k$-means as
metric projections onto 1-skeleta or $k$-skeleta in Wasserstein space, and we
define a new and more general notion of polymeans. This geometric
characterization via metric projections applies equally to sample and
population means, and we use it to establish asymptotic properties of polymeans
such as consistency and asymptotic normality.
