Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Neural Architecture Search on ImageNet in Four GPU Hours: A
  Theoretically Inspired Perspective   Neural Architecture Search (NAS) has been explosively studied to automate the
discovery of top-performer neural networks. Current works require heavy
training of supernet or intensive architecture evaluations, thus suffering from
heavy resource consumption and often incurring search bias due to truncated
training or approximations. Can we select the best neural architectures without
involving any training and eliminate a drastic portion of the search cost? We
provide an affirmative answer, by proposing a novel framework called
training-free neural architecture search (TE-NAS). TE-NAS ranks architectures
by analyzing the spectrum of the neural tangent kernel (NTK) and the number of
linear regions in the input space. Both are motivated by recent theory advances
in deep networks and can be computed without any training and any label. We
show that: (1) these two measurements imply the trainability and expressivity
of a neural network; (2) they strongly correlate with the network's test
accuracy. Further on, we design a pruning-based NAS mechanism to achieve a more
flexible and superior trade-off between the trainability and expressivity
during the search. In NAS-Bench-201 and DARTS search spaces, TE-NAS completes
high-quality search but only costs 0.5 and 4 GPU hours with one 1080Ti on
CIFAR-10 and ImageNet, respectively. We hope our work inspires more attempts in
bridging the theoretical findings of deep networks and practical impacts in
real NAS applications. Code is available at:
https://github.com/VITA-Group/TENAS.

24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
16
The categories of this paper are: stat
8
The categories of this paper are: hep-ex
25
The categories of this paper are: patt-sol
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys