Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Understanding Feature Transfer Through Representation Alignment   Training with the true labels of a dataset as opposed to randomized labels
leads to faster optimization and better generalization. This difference is
attributed to a notion of alignment between inputs and labels in natural
datasets. We find that training neural networks with different architectures
and optimizers on random or true labels enforces the same relationship between
the hidden representations and the training labels, elucidating why neural
network representations have been so successful for transfer. We first
highlight why aligned features promote transfer and show in a classic synthetic
transfer problem that alignment is the determining factor for positive and
negative transfer to similar and dissimilar tasks. We then investigate a
variety of neural network architectures and find that (a) alignment emerges
across a variety of different architectures and optimizers, with more alignment
arising from depth (b) alignment increases for layers closer to the output and
(c) existing high-performance deep CNNs exhibit high levels of alignment.

35
The categories of this paper are: mtrl-th
29
The categories of this paper are: funct-an
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn
7
The categories of this paper are: hep-th
30
The categories of this paper are: q-alg
2
The categories of this paper are: math
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th