  Deep neural networks are inherently opaque and challenging to interpret.
Unlike hand-crafted feature-based models, we struggle to comprehend the
concepts learned and how they interact within these models. This understanding
is crucial not only for debugging purposes but also for ensuring fairness in
ethical decision-making. In our study, we conduct a post-hoc functional
interpretability analysis of pretrained speech models using the probing
framework [1]. Specifically, we analyze utterance-level representations of
speech models trained for various tasks such as speaker recognition and dialect
identification. We conduct layer and neuron-wise analyses, probing for speaker,
language, and channel properties. Our study aims to answer the following
questions: i) what information is captured within the representations? ii) how
is it represented and distributed? and iii) can we identify a minimal subset of
the network that possesses this information?
  Our results reveal several novel findings, including: i) channel and gender
information are distributed across the network, ii) the information is
redundantly available in neurons with respect to a task, iii) complex
properties such as dialectal information are encoded only in the task-oriented
pretrained network, iv) and is localised in the upper layers, v) we can extract
a minimal subset of neurons encoding the pre-defined property, vi) salient
neurons are sometimes shared between properties, vii) our analysis highlights
the presence of biases (for example gender) in the network. Our
cross-architectural comparison indicates that: i) the pretrained models capture
speaker-invariant information, and ii) CNN models are competitive with
Transformer models in encoding various understudied properties.
