Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Interactive Knowledge Distillation   Knowledge distillation is a standard teacher-student learning framework to
train a light-weight student network under the guidance of a well-trained large
teacher network. As an effective teaching strategy, interactive teaching has
been widely employed at school to motivate students, in which teachers not only
provide knowledge but also give constructive feedback to students upon their
responses, to improve their learning performance. In this work, we propose an
InterActive Knowledge Distillation (IAKD) scheme to leverage the interactive
teaching strategy for efficient knowledge distillation. In the distillation
process, the interaction between teacher and student networks is implemented by
a swapping-in operation: randomly replacing the blocks in the student network
with the corresponding blocks in the teacher network. In the way, we directly
involve the teacher's powerful feature transformation ability to largely boost
the student's performance. Experiments with typical settings of teacher-student
networks demonstrate that the student networks trained by our IAKD achieve
better performance than those trained by conventional knowledge distillation
methods on diverse image classification datasets.

7
The categories of this paper are: hep-th
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
35
The categories of this paper are: mtrl-th