Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Interpolatron: Interpolation or Extrapolation Schemes to Accelerate
  Optimization for Deep Neural Networks   In this paper we explore acceleration techniques for large scale nonconvex
optimization problems with special focuses on deep neural networks. The
extrapolation scheme is a classical approach for accelerating stochastic
gradient descent for convex optimization, but it does not work well for
nonconvex optimization typically. Alternatively, we propose an interpolation
scheme to accelerate nonconvex optimization and call the method Interpolatron.
We explain motivation behind Interpolatron and conduct a thorough empirical
analysis. Empirical results on DNNs of great depths (e.g., 98-layer ResNet and
200-layer ResNet) on CIFAR-10 and ImageNet show that Interpolatron can converge
much faster than the state-of-the-art methods such as the SGD with momentum and
Adam. Furthermore, Anderson's acceleration, in which mixing coefficients are
computed by least-squares estimation, can also be used to improve the
performance. Both Interpolatron and Anderson's acceleration are easy to
implement and tune. We also show that Interpolatron has linear convergence rate
under certain regularity assumptions.

11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
36
The categories of this paper are: cmp-lg
7
The categories of this paper are: hep-th
25
The categories of this paper are: patt-sol
18
The categories of this paper are: econ
30
The categories of this paper are: q-alg
8
The categories of this paper are: hep-ex
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
4
The categories of this paper are: cond-mat