Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Contrastive Neural Architecture Search with Neural Architecture
  Comparators   One of the key steps in Neural Architecture Search (NAS) is to estimate the
performance of candidate architectures. Existing methods either directly use
the validation performance or learn a predictor to estimate the performance.
However, these methods can be either computationally expensive or very
inaccurate, which may severely affect the search efficiency and performance.
Moreover, as it is very difficult to annotate architectures with accurate
performance on specific tasks, learning a promising performance predictor is
often non-trivial due to the lack of labeled data. In this paper, we argue that
it may not be necessary to estimate the absolute performance for NAS. On the
contrary, we may need only to understand whether an architecture is better than
a baseline one. However, how to exploit this comparison information as the
reward and how to well use the limited labeled data remains two great
challenges. In this paper, we propose a novel Contrastive Neural Architecture
Search (CTNAS) method which performs architecture search by taking the
comparison results between architectures as the reward. Specifically, we design
and learn a Neural Architecture Comparator (NAC) to compute the probability of
candidate architectures being better than a baseline one. Moreover, we present
a baseline updating scheme to improve the baseline iteratively in a curriculum
learning manner. More critically, we theoretically show that learning NAC is
equivalent to optimizing the ranking over architectures. Extensive experiments
in three search spaces demonstrate the superiority of our CTNAS over existing
methods.

22
The categories of this paper are: chao-dyn
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
19
The categories of this paper are: eess
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math