Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Intra-Utterance Similarity Preserving Knowledge Distillation for Audio
  Tagging   Knowledge Distillation (KD) is a popular area of research for reducing the
size of large models while still maintaining good performance. The outputs of
larger teacher models are used to guide the training of smaller student models.
Given the repetitive nature of acoustic events, we propose to leverage this
information to regulate the KD training for Audio Tagging. This novel KD
method, "Intra-Utterance Similarity Preserving KD" (IUSP), shows promising
results for the audio tagging task. It is motivated by the previously published
KD method: "Similarity Preserving KD" (SP). However, instead of preserving the
pairwise similarities between inputs within a mini-batch, our method preserves
the pairwise similarities between the frames of a single input utterance. Our
proposed KD method, IUSP, shows consistent improvements over SP across student
models of different sizes on the DCASE 2019 Task 5 dataset for audio tagging.
There is a 27.1% to 122.4% percent increase in improvement of micro AUPRC over
the baseline relative to SP's improvement of over the baseline.

21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
16
The categories of this paper are: stat
31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
3
The categories of this paper are: physics
35
The categories of this paper are: mtrl-th
30
The categories of this paper are: q-alg
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
36
The categories of this paper are: cmp-lg