Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Mutual Learning   Model distillation is an effective and widely used technique to transfer
knowledge from a teacher to a student network. The typical application is to
transfer from a powerful large network or ensemble to a small network, that is
better suited to low-memory or fast execution requirements. In this paper, we
present a deep mutual learning (DML) strategy where, rather than one way
transfer between a static pre-defined teacher and a student, an ensemble of
students learn collaboratively and teach each other throughout the training
process. Our experiments show that a variety of network architectures benefit
from mutual learning and achieve compelling results on CIFAR-100 recognition
and Market-1501 person re-identification benchmarks. Surprisingly, it is
revealed that no prior powerful teacher network is necessary -- mutual learning
of a collection of simple student networks works, and moreover outperforms
distillation from a more powerful yet static teacher.

11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con
4
The categories of this paper are: cond-mat
17
The categories of this paper are: q-fin
15
The categories of this paper are: nucl-ex
9
The categories of this paper are: nlin
23
The categories of this paper are: solv-int
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
28
The categories of this paper are: alg-geom
22
The categories of this paper are: chao-dyn
3
The categories of this paper are: physics
16
The categories of this paper are: stat
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph