  Regression models, in which the observed features $X \in \R^p$ and the
response $Y \in \R$ depend, jointly, on a lower dimensional, unobserved, latent
vector $Z \in \R^K$, with $K< p$, are popular in a large array of applications,
and mainly used for predicting a response from correlated features. In
contrast, methodology and theory for inference on the regression coefficient
$\beta$ relating $Y$ to $Z$ are scarce, since typically the un-observable
factor $Z$ is hard to interpret. Furthermore, the determination of the
asymptotic variance of an estimator of $\beta$ is a long-standing problem, with
solutions known only in a few particular cases. To address some of these
outstanding questions, we develop inferential tools for $\beta$ in a class of
factor regression models in which the observed features are signed mixtures of
the latent factors. The model specifications are practically desirable, in a
large array of applications, render interpretability to the components of $Z$,
and are sufficient for parameter identifiability. Without assuming that the
number of latent factors $K$ or the structure of the mixture is known in
advance, we construct computationally efficient estimators of $\beta$, along
with estimators of other important model parameters. We benchmark the rate of
convergence of $\beta$ by first establishing its $\ell_2$-norm minimax lower
bound, and show that our proposed estimator is minimax-rate adaptive. Our main
contribution is the provision of a unified analysis of the component-wise
Gaussian asymptotic distribution of $\wh \beta$ and, especially, the derivation
of a closed form expression of its asymptotic variance, together with
consistent variance estimators. The resulting inferential tools can be used
when both $K$ and $p$ are independent of the sample size $n$, and when both, or
either, $p$ and $K$ vary with $n$, while allowing for $p > n$.
