Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Identifying and attacking the saddle point problem in high-dimensional
  non-convex optimization   A central challenge to many fields of science and engineering involves
minimizing non-convex error functions over continuous, high dimensional spaces.
Gradient descent or quasi-Newton methods are almost ubiquitously used to
perform such minimizations, and it is often thought that a main source of
difficulty for these local methods to find the global minimum is the
proliferation of local minima with much higher error than the global minimum.
Here we argue, based on results from statistical physics, random matrix theory,
neural network theory, and empirical evidence, that a deeper and more profound
difficulty originates from the proliferation of saddle points, not local
minima, especially in high dimensional problems of practical interest. Such
saddle points are surrounded by high error plateaus that can dramatically slow
down learning, and give the illusory impression of the existence of a local
minimum. Motivated by these arguments, we propose a new approach to
second-order optimization, the saddle-free Newton method, that can rapidly
escape high dimensional saddle points, unlike gradient descent and quasi-Newton
methods. We apply this algorithm to deep or recurrent neural network training,
and provide numerical evidence for its superior optimization performance.

28
The categories of this paper are: alg-geom
37
The categories of this paper are: supr-con
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
25
The categories of this paper are: patt-sol
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
36
The categories of this paper are: cmp-lg
17
The categories of this paper are: q-fin
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn