Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Distilling Audio-Visual Knowledge by Compositional Contrastive Learning   Having access to multi-modal cues (e.g. vision and audio) empowers some
cognitive tasks to be done faster compared to learning from a single modality.
In this work, we propose to transfer knowledge across heterogeneous modalities,
even though these data modalities may not be semantically correlated. Rather
than directly aligning the representations of different modalities, we compose
audio, image, and video representations across modalities to uncover richer
multi-modal knowledge. Our main idea is to learn a compositional embedding that
closes the cross-modal semantic gap and captures the task-relevant semantics,
which facilitates pulling together representations across modalities by
compositional contrastive learning. We establish a new, comprehensive
multi-modal distillation benchmark on three video datasets: UCF101,
ActivityNet, and VGGSound. Moreover, we demonstrate that our model
significantly outperforms a variety of existing knowledge distillation methods
in transferring audio-visual knowledge to improve video representation
learning. Code is released here: https://github.com/yanbeic/CCL.

16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
2
The categories of this paper are: math
19
The categories of this paper are: eess
37
The categories of this paper are: supr-con
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
22
The categories of this paper are: chao-dyn
21
The categories of this paper are: adap-org
0
The categories of this paper are: hep-ph
10
The categories of this paper are: q-bio
12
The categories of this paper are: nucl-th