  This paper proposes reverberation as supervision (RAS), a novel unsupervised
loss function for single-channel reverberant speech separation. Prior methods
for unsupervised separation required the synthesis of mixtures of mixtures or
assumed the existence of a teacher model, making them difficult to consider as
potential methods explaining the emergence of separation abilities in an
animal's auditory system. We assume the availability of two-channel mixtures at
training time, and train a neural network to separate the sources given one of
the channels as input such that the other channel may be predicted from the
separated sources. As the relationship between the room impulse responses
(RIRs) of each channel depends on the locations of the sources, which are
unknown to the network, the network cannot rely on learning that relationship.
Instead, our proposed loss function fits each of the separated sources to the
mixture in the target channel via Wiener filtering, and compares the resulting
mixture to the ground-truth one. We show that minimizing the scale-invariant
signal-to-distortion ratio (SI-SDR) of the predicted right-channel mixture with
respect to the ground truth implicitly guides the network towards separating
the left-channel sources. On a semi-supervised reverberant speech separation
task based on the WHAMR! dataset, using training data where just 5% (resp.,
10%) of the mixtures are labeled with associated isolated sources, we achieve
70% (resp., 78%) of the SI-SDR improvement obtained when training with
supervision on the full training set, while a model trained only on the labeled
data obtains 43% (resp., 45%).
