Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Does Visual Self-Supervision Improve Learning of Speech Representations
  for Emotion Recognition?   Self-supervised learning has attracted plenty of recent research interest.
However, most works for self-supervision in speech are typically unimodal and
there has been limited work that studies the interaction between audio and
visual modalities for cross-modal self-supervision. This work (1) investigates
visual self-supervision via face reconstruction to guide the learning of audio
representations; (2) proposes an audio-only self-supervision approach for
speech representation learning; (3) shows that a multi-task combination of the
proposed visual and audio self-supervision is beneficial for learning richer
features that are more robust in noisy conditions; (4) shows that
self-supervised pretraining can outperform fully supervised training and is
especially useful to prevent overfitting on smaller sized datasets. We evaluate
our learned audio representations for discrete emotion recognition, continuous
affect recognition and automatic speech recognition. We outperform existing
self-supervised methods for all tested downstream tasks. Our results
demonstrate the potential of visual self-supervision for audio feature learning
and suggest that joint visual and audio self-supervision leads to more
informative audio representations for speech and emotion recognition.

34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
9
The categories of this paper are: nlin
21
The categories of this paper are: adap-org
30
The categories of this paper are: q-alg
12
The categories of this paper are: nucl-th
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
35
The categories of this paper are: mtrl-th
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas