  A robust multichannel speaker diarization and separation system is proposed
by exploiting the spatio-temporal activity of the speakers. The system is
realized in a hybrid architecture that combines the array signal processing
units and the deep learning units. For speaker diarization, a spatial coherence
matrix across time frames is computed based on the whitened relative transfer
functions (wRTFs) of the microphone array. This serves as a robust feature for
subsequent machine learning without the need for prior knowledge of the array
configuration. A computationally efficient Spatial Activity-driven Speaker
Diarization network (SASDnet) is constructed to estimate the speaker activity
directly from the spatial coherence matrix. For speaker separation, we propose
the Global and Local Activity-driven Speaker Extraction network (GLASEnet) to
separate speaker signals via speaker-specific global and local spatial activity
functions. The local spatial activity functions depend on the coherence between
the wRTFs of each time-frequency bin and the target speaker-dominant bins. The
global spatial activity functions are computed from the global spatial
coherence functions based on frequency-averaged local spatial activity
functions. Experimental results have demonstrated superior speaker,
diarization, counting, and separation performance achieved by the proposed
system with low computational complexity compared to the pre-selected
baselines.
