  The subspace-based techniques are widely utilized to estimate the parameters
of sums of complex sinusoids corrupted by noise, and the zoom ESPRIT algorithm
utilizes the zoom technique to apply the ESPRIT to a narrow frequency band to
improve the accuracy of frequency estimation. However, the Gaussian noise
becomes non-Gaussian in the zoomed baseband after being filtered by a low-pass
filter, and thus has an unknown covariance matrix. However, most exiting
algorithms for model order estimation performs poorly for the case of colored
noise with unknown covariance matrix. In order to accurately estimate the
dimension of the signal subspace for the zoom ESPRIT algorithm, this paper
proposes a novel strategy to estimate the number of signals for the case of
colored noise with unknown covariance matrix. The proposed strategy is based on
the analysis of the behavior of information theoretic criteria utilized in
model order selection. Firstly, a first criterion is defined as the ratio of
the current eigenvalue and the mean of the next ones, and its properties is
analyzed with respect to the over-modeling and under-modeling. Secondly, a
novel second criterion is designed as the ratio of the current value and the
next value of the first criterion, and its properties is also analyzed with
respect to the over-modeling and under-modeling. Then, a novel signal number
estimation method is proposed by combining the second criterion with the first
criterion to check whether the eigenvalue being tested is arising from a signal
or from noise. The resulted signal number estimation method is called as the
double criterion-based estimator as it utilizes two criteria to separate the
signal eigenvalues from the noise eigenvalues. Finally, simulation results are
presented to illustrate the performance of the proposed double criterion-based
estimator and compare it with the existing methods.
