Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An empirical analysis of the optimization of deep network loss surfaces   The success of deep neural networks hinges on our ability to accurately and
efficiently optimize high-dimensional, non-convex functions. In this paper, we
empirically investigate the loss functions of state-of-the-art networks, and
how commonly-used stochastic gradient descent variants optimize these loss
functions. To do this, we visualize the loss function by projecting them down
to low-dimensional spaces chosen based on the convergence points of different
optimization algorithms. Our observations suggest that optimization algorithms
encounter and choose different descent directions at many saddle points to find
different final weights. Based on consistency we observe across re-runs of the
same stochastic optimization algorithm, we hypothesize that each optimization
algorithm makes characteristic choices at these saddle points.

15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
26
The categories of this paper are: bayes-an
4
The categories of this paper are: cond-mat
2
The categories of this paper are: math
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn