Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Deep Representations with Probabilistic Knowledge Transfer   Knowledge Transfer (KT) techniques tackle the problem of transferring the
knowledge from a large and complex neural network into a smaller and faster
one. However, existing KT methods are tailored towards classification tasks and
they cannot be used efficiently for other representation learning tasks. In
this paper a novel knowledge transfer technique, that is capable of training a
student model that maintains the same amount of mutual information between the
learned representation and a set of (possible unknown) labels as the teacher
model, is proposed. Apart from outperforming existing KT techniques, the
proposed method allows for overcoming several limitations of existing methods
providing new insight into KT as well as novel KT applications, ranging from
knowledge transfer from handcrafted feature extractors to {cross-modal} KT from
the textual modality into the representation extracted from the visual modality
of the data.

18
The categories of this paper are: econ
12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
26
The categories of this paper are: bayes-an
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph
23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat