Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multilayer Lookahead: a Nested Version of Lookahead   In recent years, SGD and its variants have become the standard tool to train
Deep Neural Networks. In this paper, we focus on the recently proposed variant
Lookahead, which improves upon SGD in a wide range of applications. Following
this success, we study an extension of this algorithm, the \emph{Multilayer
Lookahead} optimizer, which recursively wraps Lookahead around itself. We prove
the convergence of Multilayer Lookahead with two layers to a stationary point
of smooth non-convex functions with $O(\frac{1}{\sqrt{T}})$ rate. We also
justify the improved generalization of both Lookahead over SGD, and of
Multilayer Lookahead over Lookahead, by showing how they amplify the implicit
regularization effect of SGD. We empirically verify our results and show that
Multilayer Lookahead outperforms Lookahead on CIFAR-10 and CIFAR-100
classification tasks, and on GANs training on the MNIST dataset.

10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
13
The categories of this paper are: quant-ph
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
18
The categories of this paper are: econ
16
The categories of this paper are: stat
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
22
The categories of this paper are: chao-dyn
4
The categories of this paper are: cond-mat