  Previous studies demonstrate the impressive performance of residual neural
networks (ResNet) in speaker verification. The ResNet models treat the time and
frequency dimensions equally. They follow the default stride configuration
designed for image recognition, where the horizontal and vertical axes exhibit
similarities. This approach ignores the fact that time and frequency are
asymmetric in speech representation. In this paper, we address this issue and
look for optimal stride configurations specifically tailored for speaker
verification. We represent the stride space on a trellis diagram, and conduct a
systematic study on the impact of temporal and frequency resolutions on the
performance and further identify two optimal points, namely Golden Gemini,
which serves as a guiding principle for designing 2D ResNet-based speaker
verification models. By following the principle, a state-of-the-art ResNet
baseline model gains a significant performance improvement on VoxCeleb, SITW,
and CNCeleb datasets with 7.70%/11.76% average EER/minDCF reductions,
respectively, across different network depths (ResNet18, 34, 50, and 101),
while reducing the number of parameters by 16.5% and FLOPs by 4.1%. We refer to
it as Gemini ResNet. Further investigation reveals the efficacy of the proposed
Golden Gemini operating points across various training conditions and
architectures. Furthermore, we present a new benchmark, namely the Gemini
DF-ResNet, using a cutting-edge model.
