  Speech enhancement employing deep neural networks (DNNs) for denoising are
called deep noise suppression (DNS). During training, DNS methods are typically
trained with mean squared error (MSE) type loss functions, which do not
guarantee good perceptual quality. Perceptual evaluation of speech quality
(PESQ) is a widely used metric for evaluating speech quality. However, the
original PESQ algorithm is non-differentiable, and therefore cannot directly be
used as optimization criterion for gradient-based learning. In this work, we
propose an end-to-end non-intrusive PESQNet DNN to estimate the PESQ scores of
the enhanced speech signal. Thus, by providing a reference-free perceptual
loss, it serves as a mediator towards the DNS training, allowing to maximize
the PESQ score of the enhanced speech signal. We illustrate the potential of
our proposed PESQNet-mediated training on the basis of an already strong
baseline DNS. As further novelty, we propose to train the DNS and the PESQNet
alternatingly to keep the PESQNet up-to-date and perform well specifically for
the DNS under training. Our proposed method is compared to the same DNS trained
with MSE-based loss for joint denoising and dereverberation, and the
Interspeech 2021 DNS Challenge baseline. Detailed analysis shows that the
PESQNet mediation can further increase the DNS performance by about 0.1 PESQ
points on synthetic test data and by 0.03 DNSMOS points on real test data,
compared to training with the MSE-based loss. Our proposed method also
outperforms the Challenge baseline by 0.2 PESQ points on synthetic test data
and 0.1 DNSMOS points on real test data.
