Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Number of Steps Needed for Nonconvex Optimization of a Deep Learning
  Optimizer is a Rational Function of Batch Size   Recently, convergence as well as convergence rate analyses of deep learning
optimizers for nonconvex optimization have been widely studied. Meanwhile,
numerical evaluations for the optimizers have precisely clarified the
relationship between batch size and the number of steps needed for training
deep neural networks. The main contribution of this paper is to show
theoretically that the number of steps needed for nonconvex optimization of
each of the optimizers can be expressed as a rational function of batch size.
Having these rational functions leads to two particularly important facts,
which were validated numerically in previous studies. The first fact is that
there exists an optimal batch size such that the number of steps needed for
nonconvex optimization is minimized. This implies that using larger batch sizes
than the optimal batch size does not decrease the number of steps needed for
nonconvex optimization. The second fact is that the optimal batch size depends
on the optimizer. In particular, it is shown theoretically that momentum and
Adam-type optimizers can exploit larger optimal batches and further reduce the
minimum number of steps needed for nonconvex optimization than can the
stochastic gradient descent optimizer.

30
The categories of this paper are: q-alg
36
The categories of this paper are: cmp-lg
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
13
The categories of this paper are: quant-ph
9
The categories of this paper are: nlin
21
The categories of this paper are: adap-org
16
The categories of this paper are: stat
25
The categories of this paper are: patt-sol
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con