Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Nonlinearity Coefficient - Predicting Generalization in Deep Neural
  Networks   For a long time, designing neural architectures that exhibit high performance
was considered a dark art that required expert hand-tuning. One of the few
well-known guidelines for architecture design is the avoidance of exploding
gradients, though even this guideline has remained relatively vague and
circumstantial. We introduce the nonlinearity coefficient (NLC), a measurement
of the complexity of the function computed by a neural network that is based on
the magnitude of the gradient. Via an extensive empirical study, we show that
the NLC is a powerful predictor of test error and that attaining a right-sized
NLC is essential for optimal performance.
  The NLC exhibits a range of intriguing and important properties. It is
closely tied to the amount of information gained from computing a single
network gradient. It is tied to the error incurred when replacing the
nonlinearity operations in the network with linear operations. It is not
susceptible to the confounders of multiplicative scaling, additive bias and
layer width. It is stable from layer to layer. Hence, we argue that the NLC is
the first robust predictor of overfitting in deep networks.

0
The categories of this paper are: hep-ph
31
The categories of this paper are: ao-sci
6
The categories of this paper are: astro-ph
28
The categories of this paper are: alg-geom
36
The categories of this paper are: cmp-lg
17
The categories of this paper are: q-fin
24
The categories of this paper are: dg-ga
35
The categories of this paper are: mtrl-th
33
The categories of this paper are: plasm-ph
11
The categories of this paper are: hep-lat
32
The categories of this paper are: atom-ph
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys