Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Diagonal Rescaling For Neural Networks   We define a second-order neural network stochastic gradient training
algorithm whose block-diagonal structure effectively amounts to normalizing the
unit activations. Investigating why this algorithm lacks in robustness then
reveals two interesting insights. The first insight suggests a new way to scale
the stepsizes, clarifying popular algorithms such as RMSProp as well as old
neural network tricks such as fanin stepsize scaling. The second insight
stresses the practical importance of dealing with fast changes of the curvature
of the cost.

31
The categories of this paper are: ao-sci
9
The categories of this paper are: nlin
37
The categories of this paper are: supr-con
25
The categories of this paper are: patt-sol
23
The categories of this paper are: solv-int
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math
0
The categories of this paper are: hep-ph
35
The categories of this paper are: mtrl-th
30
The categories of this paper are: q-alg
4
The categories of this paper are: cond-mat
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess