Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Neural Architecture Optimization   Automatic neural architecture design has shown its potential in discovering
powerful neural network architectures. Existing methods, no matter based on
reinforcement learning or evolutionary algorithms (EA), conduct architecture
search in a discrete space, which is highly inefficient. In this paper, we
propose a simple and efficient method to automatic neural architecture design
based on continuous optimization. We call this new approach neural architecture
optimization (NAO). There are three key components in our proposed approach:
(1) An encoder embeds/maps neural network architectures into a continuous
space. (2) A predictor takes the continuous representation of a network as
input and predicts its accuracy. (3) A decoder maps a continuous representation
of a network back to its architecture. The performance predictor and the
encoder enable us to perform gradient based optimization in the continuous
space to find the embedding of a new architecture with potentially better
accuracy. Such a better embedding is then decoded to a network by the decoder.
Experiments show that the architecture discovered by our method is very
competitive for image classification task on CIFAR-10 and language modeling
task on PTB, outperforming or on par with the best results of previous
architecture search methods with a significantly reduction of computational
resources. Specifically we obtain 1.93% test set error rate for CIFAR-10 image
classification task and 56.0 test set perplexity of PTB language modeling task.
Furthermore, combined with the recent proposed weight sharing mechanism, we
discover powerful architecture on CIFAR-10 (with error rate 2.93%) and on PTB
(with test set perplexity 56.6), with very limited computational resources
(less than 10 GPU hours) for both tasks.

31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
14
The categories of this paper are: math-ph
13
The categories of this paper are: quant-ph