  Many statistical models seek relationship between variables via subspaces of
reduced dimensions. For instance, in factor models, variables are roughly
distributed around a low dimensional subspace determined by the loading matrix;
in mixed linear regression models, the coefficient vectors for different
mixtures form a subspace that captures all regression functions; in multiple
index models, the effect of covariates is summarized by the effective dimension
reduction space.
  Such subspaces are typically unknown, and good estimates are crucial for data
visualization, dimension reduction, diagnostics and estimation of unknown
parameters. Usually, we can estimate these subspaces by computing moments from
data. Often, there are many ways to estimate a subspace, by using moments of
different orders, transformed moments, etc. A natural question is: how can we
combine all these moment conditions and achieve optimality for subspace
estimation?
  In this paper, we formulate our problem as estimation of an unknown subspace
$\mathcal{S}$ of dimension $r$, given a set of overidentifying vectors $\{
\mathrm{\bf v}_\ell \}_{\ell=1}^m$ (namely $m \ge r$) that satisfy $\mathbb{E}
\mathrm{\bf v}_{\ell} \in \mathcal{S}$ and have the form $$ \mathrm{\bf v}_\ell
= \frac{1}{n} \sum_{i=1}^n \mathrm{\bf f}_\ell(\mathbf{x}_i, y_i), $$ where
data are i.i.d. and each function $\mathrm{\bf f}_\ell$ is known. By exploiting
certain covariance information related to $\mathrm{\bf v}_\ell$, our estimator
of $\mathcal{S}$ uses an optimal weighting matrix and achieves the smallest
asymptotic error, in terms of canonical angles. The analysis is based on the
generalized method of moments that is tailored to our problem. Our method is
applied to aforementioned models and distributed estimation of heterogeneous
datasets, and may be potentially extended to analyze matrix completion, neural
nets, among others.
