Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multi-Modal Emotion Detection with Transfer Learning   Automated emotion detection in speech is a challenging task due to the
complex interdependence between words and the manner in which they are spoken.
It is made more difficult by the available datasets; their small size and
incompatible labeling idiosyncrasies make it hard to build generalizable
emotion detection systems. To address these two challenges, we present a
multi-modal approach that first transfers learning from related tasks in speech
and text to produce robust neural embeddings and then uses these embeddings to
train a pLDA classifier that is able to adapt to previously unseen emotions and
domains. We begin by training a multilayer TDNN on the task of speaker
identification with the VoxCeleb corpora and then fine-tune it on the task of
emotion identification with the Crema-D corpus. Using this network, we extract
speech embeddings for Crema-D from each of its layers, generate and concatenate
text embeddings for the accompanying transcripts using a fine-tuned BERT model
and then train an LDA - pLDA classifier on the resulting dense representations.
We exhaustively evaluate the predictive power of every component: the TDNN
alone, speech embeddings from each of its layers alone, text embeddings alone
and every combination thereof. Our best variant, trained on only VoxCeleb and
Crema-D and evaluated on IEMOCAP, achieves an EER of 38.05%. Including a
portion of IEMOCAP during training produces a 5-fold averaged EER of 25.72%
(For comparison, 44.71% of the gold-label annotations include at least one
annotator who disagrees).

27
The categories of this paper are: comp-gas
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
22
The categories of this paper are: chao-dyn
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph
25
The categories of this paper are: patt-sol
33
The categories of this paper are: plasm-ph
12
The categories of this paper are: nucl-th
32
The categories of this paper are: atom-ph
21
The categories of this paper are: adap-org
29
The categories of this paper are: funct-an