  This work proposes a multichannel narrow-band speech separation network. In
the short-time Fourier transform (STFT) domain, the proposed network processes
each frequency independently, and all frequencies use a shared network. For
each frequency, the network performs end-to-end speech separation, namely
taking as input the STFT coefficients of microphone signals, and predicting the
separated STFT coefficients of multiple speakers. The proposed network learns
to cluster the frame-wise spatial/steering vectors that belong to different
speakers. It is mainly composed of three components. First, a self-attention
network. Clustering of spatial vectors shares a similar principle with the
self-attention mechanism in the sense of computing the similarity of vectors
and then aggregating similar vectors. Second, a convolutional feed-forward
network. The convolutional layers are employed for signal smoothing and
reverberation processing. Third, a novel hidden-layer normalization method,
i.e. group batch normalization (GBN), is especially designed for the proposed
narrow-band network to maintain the distribution of hidden units over
frequencies. Overall, the proposed network is named NBC2, as it is a revised
version of our previous NBC (narrow-band conformer) network. Experiments show
that 1) the proposed network outperforms other state-of-the-art methods by a
large margin, 2) the proposed GBN improves the signal-to-distortion ratio by 3
dB, relative to other normalization methods, such as batch/layer/group
normalization, 3) the proposed narrow-band network is spectrum-agnostic, as it
does not learn spectral patterns, and 4) the proposed network is indeed
performing frame clustering (demonstrated by the attention maps).
