  Despite the recent success of deep learning for many speech processing tasks,
single-microphone, speaker-independent speech separation remains challenging
for two main reasons. The first reason is the arbitrary order of the target and
masker speakers in the mixture permutation problem, and the second is the
unknown number of speakers in the mixture output dimension problem. We propose
a novel deep learning framework for speech separation that addresses both of
these issues. We use a neural network to project the time-frequency
representation of the mixture signal into a high-dimensional embedding space. A
reference point attractor is created in the embedding space to represent each
speaker which is defined as the centroid of the speaker in the embedding space.
The time-frequency embeddings of each speaker are then forced to cluster around
the corresponding attractor point which is used to determine the time-frequency
assignment of the speaker. We propose three methods for finding the attractors
for each source in the embedding space and compare their advantages and
limitations. The objective function for the network is standard signal
reconstruction error which enables end-to-end operation during both training
and test phases. We evaluated our system using the Wall Street Journal dataset
WSJ0 on two and three speaker mixtures and report comparable or better
performance than other state-of-the-art deep learning methods for speech
separation.
