  We propose TF-GridNet for speech separation. The model is a novel deep neural
network (DNN) integrating full- and sub-band modeling in the time-frequency
(T-F) domain. It stacks several blocks, each consisting of an intra-frame
full-band module, a sub-band temporal module, and a cross-frame self-attention
module. It is trained to perform complex spectral mapping, where the real and
imaginary (RI) components of input signals are stacked as features to predict
target RI components. We first evaluate it on monaural anechoic speaker
separation. Without using data augmentation and dynamic mixing, it obtains a
state-of-the-art 23.5 dB improvement in scale-invariant signal-to-distortion
ratio (SI-SDR) on WSJ0-2mix, a standard dataset for two-speaker separation. To
show its robustness to noise and reverberation, we evaluate it on monaural
reverberant speaker separation using the SMS-WSJ dataset and on
noisy-reverberant speaker separation using WHAMR!, and obtain state-of-the-art
performance on both datasets. We then extend TF-GridNet to multi-microphone
conditions through multi-microphone complex spectral mapping, and integrate it
into a two-DNN system with a beamformer in between (named as MISO-BF-MISO in
earlier studies), where the beamformer proposed in this paper is a novel
multi-frame Wiener filter computed based on the outputs of the first DNN.
State-of-the-art performance is obtained on the multi-channel tasks of SMS-WSJ
and WHAMR!. Besides speaker separation, we apply the proposed algorithms to
speech dereverberation and noisy-reverberant speech enhancement.
State-of-the-art performance is obtained on a dereverberation dataset and on
the dataset of the recent L3DAS22 multi-channel speech enhancement challenge.
