Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep
  Learning   In this paper, a novel second-order method called NG+ is proposed. By
following the rule ``the shape of the gradient equals the shape of the
parameter", we define a generalized fisher information matrix (GFIM) using the
products of gradients in the matrix form rather than the traditional
vectorization. Then, our generalized natural gradient direction is simply the
inverse of the GFIM multiplies the gradient in the matrix form. Moreover, the
GFIM and its inverse keeps the same for multiple steps so that the
computational cost can be controlled and is comparable with the first-order
methods. A global convergence is established under some mild conditions and a
regret bound is also given for the online learning setting. Numerical results
on image classification with ResNet50, quantum chemistry modeling with Schnet,
neural machine translation with Transformer and recommendation system with DLRM
illustrate that GN+ is competitive with the state-of-the-art methods.

4
The categories of this paper are: cond-mat
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
22
The categories of this paper are: chao-dyn
37
The categories of this paper are: supr-con
36
The categories of this paper are: cmp-lg
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
11
The categories of this paper are: hep-lat
18
The categories of this paper are: econ
32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
34
The categories of this paper are: chem-ph