Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Loss Surface of Residual Networks: Ensembles and the Role of Batch
  Normalization   Deep Residual Networks present a premium in performance in comparison to
conventional networks of the same depth and are trainable at extreme depths. It
has recently been shown that Residual Networks behave like ensembles of
relatively shallow networks. We show that these ensembles are dynamic: while
initially the virtual ensemble is mostly at depths lower than half the
network's depth, as training progresses, it becomes deeper and deeper. The main
mechanism that controls the dynamic ensemble behavior is the scaling
introduced, e.g., by the Batch Normalization technique. We explain this
behavior and demonstrate the driving force behind it. As a main tool in our
analysis, we employ generalized spin glass models, which we also use in order
to study the number of critical points in the optimization of Residual
Networks.

8
The categories of this paper are: hep-ex
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
26
The categories of this paper are: bayes-an
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th
5
The categories of this paper are: gr-qc
6
The categories of this paper are: astro-ph
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas
25
The categories of this paper are: patt-sol
37
The categories of this paper are: supr-con