Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Knowledge Amalgamation from Heterogeneous Networks by Common Feature
  Learning   An increasing number of well-trained deep networks have been released online
by researchers and developers, enabling the community to reuse them in a
plug-and-play way without accessing the training annotations. However, due to
the large number of network variants, such public-available trained models are
often of different architectures, each of which being tailored for a specific
task or dataset. In this paper, we study a deep-model reusing task, where we
are given as input pre-trained networks of heterogeneous architectures
specializing in distinct tasks, as teacher models. We aim to learn a
multitalented and light-weight student model that is able to grasp the
integrated knowledge from all such heterogeneous-structure teachers, again
without accessing any human annotation. To this end, we propose a common
feature learning scheme, in which the features of all teachers are transformed
into a common space and the student is enforced to imitate them all so as to
amalgamate the intact knowledge. We test the proposed approach on a list of
benchmarks and demonstrate that the learned student is able to achieve very
promising performance, superior to those of the teachers in their specialized
tasks.

24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
13
The categories of this paper are: quant-ph
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
4
The categories of this paper are: cond-mat
2
The categories of this paper are: math
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an