  High-fidelity multi-singer singing voice synthesis is challenging for neural
vocoder due to the singing voice data shortage, limited singer generalization,
and large computational cost. Existing open corpora could not meet requirements
for high-fidelity singing voice synthesis because of the scale and quality
weaknesses. Previous vocoders have difficulty in multi-singer modeling, and a
distinct degradation emerges when conducting unseen singer singing voice
generation. To accelerate singing voice researches in the community, we release
a large-scale, multi-singer Chinese singing voice dataset OpenSinger. To tackle
the difficulty in unseen singer modeling, we propose Multi-Singer, a fast
multi-singer vocoder with generative adversarial networks. Specifically, 1)
Multi-Singer uses a multi-band generator to speed up both training and
inference procedure. 2) to capture and rebuild singer identity from the
acoustic feature (i.e., mel-spectrogram), Multi-Singer adopts a singer
conditional discriminator and conditional adversarial training objective. 3) to
supervise the reconstruction of singer identity in the spectrum envelopes in
frequency domain, we propose an auxiliary singer perceptual loss. The joint
training approach effectively works in GANs for multi-singer voices modeling.
Experimental results verify the effectiveness of OpenSinger and show that
Multi-Singer improves unseen singer singing voices modeling in both speed and
quality over previous methods. The further experiment proves that combined with
FastSpeech 2 as the acoustic model, Multi-Singer achieves strong robustness in
the multi-singer singing voice synthesis pipeline. Samples are available at
https://Multi-Singer.github.io/
