Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Controlling Model Complexity in Probabilistic Model-Based Dynamic
  Optimization of Neural Network Structures   A method of simultaneously optimizing both the structure of neural networks
and the connection weights in a single training loop can reduce the enormous
computational cost of neural architecture search. We focus on the probabilistic
model-based dynamic neural network structure optimization that considers the
probability distribution of structure parameters and simultaneously optimizes
both the distribution parameters and connection weights based on gradient
methods. Since the existing algorithm searches for the structures that only
minimize the training loss, this method might find overly complicated
structures. In this paper, we propose the introduction of a penalty term to
control the model complexity of obtained structures. We formulate a penalty
term using the number of weights or units and derive its analytical natural
gradient. The proposed method minimizes the objective function injected the
penalty term based on the stochastic gradient descent. We apply the proposed
method in the unit selection of a fully-connected neural network and the
connection selection of a convolutional neural network. The experimental
results show that the proposed method can control model complexity while
maintaining performance.

23
The categories of this paper are: solv-int
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
11
The categories of this paper are: hep-lat
4
The categories of this paper are: cond-mat
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
15
The categories of this paper are: nucl-ex
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph