Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Which Minimizer Does My Neural Network Converge To?   The loss surface of an overparameterized neural network (NN) possesses many
global minima of zero training error. We explain how common variants of the
standard NN training procedure change the minimizer obtained. First, we make
explicit how the size of the initialization of a strongly overparameterized NN
affects the minimizer and can deteriorate its final test performance. We
propose a strategy to limit this effect. Then, we demonstrate that for adaptive
optimization such as AdaGrad, the obtained minimizer generally differs from the
gradient descent (GD) minimizer. This adaptive minimizer is changed further by
stochastic mini-batch training, even though in the non-adaptive case GD and
stochastic GD result in essentially the same minimizer. Lastly, we explain that
these effects remain relevant for less overparameterized NNs. While
overparameterization has its benefits, our work highlights that it induces
sources of error absent from underparameterized models, some of which can be
challenging to control.

3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
7
The categories of this paper are: hep-th
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
2
The categories of this paper are: math
5
The categories of this paper are: gr-qc
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an