  We introduce Autodecompose, a novel self-supervised generative model that
decomposes data into two semantically independent properties: the desired
property, which captures a specific aspect of the data (e.g. the voice in an
audio signal), and the context property, which aggregates all other information
(e.g. the content of the audio signal), without any labels given. Autodecompose
uses two complementary augmentations, one that manipulates the context while
preserving the desired property and the other that manipulates the desired
property while preserving the context. The augmented variants of the data are
encoded by two encoders and reconstructed by a decoder. We prove that one of
the encoders embeds the desired property while the other embeds the context
property. We apply Autodecompose to audio signals to encode sound source (human
voice) and content. We pre-trained the model on YouTube and LibriSpeech
datasets and fine-tuned in a self-supervised manner without exposing the
labels. Our results showed that, using the sound source encoder of pre-trained
Autodecompose, a linear classifier achieves F1 score of 97.6\% in recognizing
the voice of 30 speakers using only 10 seconds of labeled samples, compared to
95.7\% for supervised models. Additionally, our experiments showed that
Autodecompose is robust against overfitting even when a large model is
pre-trained on a small dataset. A large Autodecompose model was pre-trained
from scratch on 60 seconds of audio from 3 speakers achieved over 98.5\% F1
score in recognizing those three speakers in other unseen utterances. We
finally show that the context encoder embeds information about the content of
the speech and ignores the sound source information.
  Our sample code for training the model, as well as examples for using the
pre-trained models are available here:
\url{https://github.com/rezabonyadi/autodecompose}
