Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multilevel Initialization for Layer-Parallel Deep Neural Network
  Training   This paper investigates multilevel initialization strategies for training
very deep neural networks with a layer-parallel multigrid solver. The scheme is
based on the continuous interpretation of the training problem as a problem of
optimal control, in which neural networks are represented as discretizations of
time-dependent ordinary differential equations. A key goal is to develop a
method able to intelligently initialize the network parameters for the very
deep networks enabled by scalable layer-parallel training. To do this, we apply
a refinement strategy across the time domain, that is equivalent to refining in
the layer dimension. The resulting refinements create deep networks, with good
initializations for the network parameters coming from the coarser trained
networks. We investigate the effectiveness of such multilevel "nested
iteration" strategies for network training, showing supporting numerical
evidence of reduced run time for equivalent accuracy. In addition, we study
whether the initialization strategies provide a regularizing effect on the
overall training process and reduce sensitivity to hyperparameters and
randomness in initial network parameters.

9
The categories of this paper are: nlin
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
13
The categories of this paper are: quant-ph
37
The categories of this paper are: supr-con
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol