  Perceptual sound matching (PSM) aims to find the input parameters to a
synthesizer so as to best imitate an audio target. Deep learning for PSM
optimizes a neural network to analyze and reconstruct prerecorded samples. In
this context, our article addresses the problem of designing a suitable loss
function when the training set is generated by a differentiable synthesizer.
Our main contribution is perceptual-neural-physical loss (PNP), which aims at
addressing a tradeoff between perceptual relevance and computational
efficiency. The key idea behind PNP is to linearize the effect of synthesis
parameters upon auditory features in the vicinity of each training sample. The
linearization procedure is massively paralellizable, can be precomputed, and
offers a 100-fold speedup during gradient descent compared to differentiable
digital signal processing (DDSP). We demonstrate PNP on two datasets of
nonstationary sounds: an AM/FM arpeggiator and a physical model of rectangular
membranes. We show that PNP is able to accelerate DDSP with joint
time-frequency scattering transform (JTFS) as auditory feature, while
preserving its perceptual fidelity. Additionally, we evaluate the impact of
other design choices in PSM: parameter rescaling, pretraining, auditory
representation, and gradient clipping. We report state-of-the-art results on
both datasets and find that PNP-accelerated JTFS has greater influence on PSM
performance than any other design choice.
