Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion
  Recognition?   Humans express their emotions via facial expressions, voice intonation and
word choices. To infer the nature of the underlying emotion, recognition models
may use a single modality, such as vision, audio, and text, or a combination of
modalities. Generally, models that fuse complementary information from multiple
modalities outperform their uni-modal counterparts. However, a successful model
that fuses modalities requires components that can effectively aggregate
task-relevant information from each modality. As cross-modal attention is seen
as an effective mechanism for multi-modal fusion, in this paper we quantify the
gain that such a mechanism brings compared to the corresponding self-attention
mechanism. To this end, we implement and compare a cross-attention and a
self-attention model. In addition to attention, each model uses convolutional
layers for local feature extraction and recurrent layers for global sequential
modelling. We compare the models using different modality combinations for a
7-class emotion classification task using the IEMOCAP dataset. Experimental
results indicate that albeit both models improve upon the state-of-the-art in
terms of weighted and unweighted accuracy for tri- and bi-modal configurations,
their performance is generally statistically comparable. The code to replicate
the experiments is available at https://github.com/smartcameras/SelfCrossAttn

21
The categories of this paper are: adap-org
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
33
The categories of this paper are: plasm-ph
19
The categories of this paper are: eess
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph