  We present a detailed quantitative analysis of spectral correlations in the
Sachdev-Ye-Kitaev (SYK) model. We find that the deviations from universal
Random Matrix Theory (RMT) behavior are due to a small number of
long-wavelength fluctuations from one realization of the ensemble to the next
one. These modes can be parameterized effectively in terms of Q-Hermite
orthogonal polynomials, the main contribution being the scale fluctuations for
which we give a simple estimate. Our numerical results for $N=32$ show that
only the lowest eight polynomials are needed to eliminate the nonuniversal part
of the spectral fluctuations. The covariance matrix of the coefficients of this
expansion is obtained analytically from low-order double-trace moments. We
evaluate the covariance matrix of the first six moments and find that it agrees
with the numerics. We also analyze the spectral correlations using a nonlinear
$\sigma$-model, which is derived through a Fierz transformation, and evaluate
the one and two-point spectral correlator to two-loop order. The wide
correlator is given by the sum of the universal RMT result and corrections
whose lowest-order term corresponds to scale fluctuations. However, the loop
expansion of the $\sigma$-model results in an ill-behaved expansion of the
resolvent, and it gives universal RMT fluctuations not only for $q=4$ but also
for the $q=2$ SYK model while the correct result in this case should have been
Poisson statistics. We analyze the number variance and spectral form factor for
$N=32$ and $q=4$ numerically. We show that the quadratic deviation of the
number variance for large energies appears as a peak for small times in the
spectral form factor. After eliminating the long-wavelength fluctuations, we
find quantitative agreement with RMT up to an exponentially large number of
level spacings or exponentially short times, respectively.
