Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Understanding the wiring evolution in differentiable neural architecture
  search   Controversy exists on whether differentiable neural architecture search
methods discover wiring topology effectively. To understand how wiring topology
evolves, we study the underlying mechanism of several existing differentiable
NAS frameworks. Our investigation is motivated by three observed searching
patterns of differentiable NAS: 1) they search by growing instead of pruning;
2) wider networks are more preferred than deeper ones; 3) no edges are selected
in bi-level optimization. To anatomize these phenomena, we propose a unified
view on searching algorithms of existing frameworks, transferring the global
optimization to local cost minimization. Based on this reformulation, we
conduct empirical and theoretical analyses, revealing implicit inductive biases
in the cost's assignment mechanism and evolution dynamics that cause the
observed phenomena. These biases indicate strong discrimination towards certain
topologies. To this end, we pose questions that future differentiable methods
for neural wiring discovery need to confront, hoping to evoke a discussion and
rethinking on how much bias has been enforced implicitly in existing NAS
methods.

11
The categories of this paper are: hep-lat
29
The categories of this paper are: funct-an
31
The categories of this paper are: ao-sci
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
24
The categories of this paper are: dg-ga
10
The categories of this paper are: q-bio
23
The categories of this paper are: solv-int
2
The categories of this paper are: math
22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th