Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Fast Approximate Natural Gradient Descent in a Kronecker-factored
  Eigenbasis   Optimization algorithms that leverage gradient covariance information, such
as variants of natural gradient descent (Amari, 1998), offer the prospect of
yielding more effective descent directions. For models with many parameters,
the covariance matrix they are based on becomes gigantic, making them
inapplicable in their original form. This has motivated research into both
simple diagonal approximations and more sophisticated factored approximations
such as KFAC (Heskes, 2000; Martens & Grosse, 2015; Grosse & Martens, 2016). In
the present work we draw inspiration from both to propose a novel approximation
that is provably better than KFAC and amendable to cheap partial updates. It
consists in tracking a diagonal variance, not in parameter coordinates, but in
a Kronecker-factored eigenbasis, in which the diagonal approximation is likely
to be more effective. Experiments show improvements over KFAC in optimization
speed for several deep network architectures.

14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin
2
The categories of this paper are: math
21
The categories of this paper are: adap-org
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
0
The categories of this paper are: hep-ph