  Let $X_1,\ldots,X_n$ be i.i.d. centered Gaussian vectors in $\mathbb{R}^p$
with covariance $\Sigma$, and let $\hat{\Sigma}\equiv n^{-1}\sum_{i=1}^n
X_iX_i^\top$ be the sample covariance. A central object of interest in the
non-asymptotic theory of sample covariance is the spectral norm error
$||\hat{\Sigma}-\Sigma||$ of the sample covariance $\hat{\Sigma}$. In the
path-breaking work of Koltchinskii and Lounici [KL17a], the `zeroth-order'
magnitude of $||\hat{\Sigma}-\Sigma||$ is characterized by the dimension-free
two-sided estimate $\mathbb{E} \{||\hat{\Sigma}-\Sigma||/||\Sigma||\}\asymp
\sqrt{r(\Sigma)/n}+r(\Sigma)/n $, using the so-called effective rank
$r(\Sigma)\equiv \mathrm{tr}(\Sigma)/||\Sigma||$.
  The goal of this paper is to provide a dimension-free first-order
characterization for $||\hat{\Sigma}-\Sigma||$. We show that \begin{equation*}
\bigg|\frac{\mathbb{E} \{||\hat{\Sigma}-\Sigma||/||\Sigma||\}
}{\mathbb{E}\sup_{\alpha \in
[0,1]}\{(\alpha+n^{-1/2}\mathscr{G}_{\Sigma}(h;\alpha))^2-\alpha^2\}}- 1\bigg|
\leq \frac{C}{\sqrt{r(\Sigma)} }, \end{equation*} where
$\{\mathscr{G}_{\Sigma}(h;\alpha): \alpha \in [0,1]\}$ are (stochastic)
Gaussian widths over spherical slices of the (standardized) $\Sigma$-ellipsoid,
playing the role of a first-order analogue to the zeroth-order characteristic
$r(\Sigma)$. As an immediate application of the first-order characterization,
we obtain a version of the Koltchinskii-Lounici bound with optimal constants.
In the more special context of spiked covariance models, our first-order
characterization reveals a new phase transition of $||\hat{\Sigma}-\Sigma||$
that exhibits qualitatively different behavior compared to the BBP phase
transitional behavior of $||\hat{\Sigma}||$. A similar phase transition is also
proved for the associated eigenvector.
