Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Limit of the Batch Size   Large-batch training is an efficient approach for current distributed deep
learning systems. It has enabled researchers to reduce the ImageNet/ResNet-50
training from 29 hours to around 1 minute. In this paper, we focus on studying
the limit of the batch size. We think it may provide a guidance to AI
supercomputer and algorithm designers. We provide detailed numerical
optimization instructions for step-by-step comparison. Moreover, it is
important to understand the generalization and optimization performance of huge
batch training. Hoffer et al. introduced "ultra-slow diffusion" theory to
large-batch training. However, our experiments show contradictory results with
the conclusion of Hoffer et al. We provide comprehensive experimental results
and detailed analysis to study the limitations of batch size scaling and
"ultra-slow diffusion" theory. For the first time we scale the batch size on
ImageNet to at least a magnitude larger than all previous work, and provide
detailed studies on the performance of many state-of-the-art optimization
schemes under this setting. We propose an optimization recipe that is able to
improve the top-1 test accuracy by 18% compared to the baseline.

25
The categories of this paper are: patt-sol
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
32
The categories of this paper are: atom-ph
18
The categories of this paper are: econ
27
The categories of this paper are: comp-gas
30
The categories of this paper are: q-alg
15
The categories of this paper are: nucl-ex
8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
34
The categories of this paper are: chem-ph
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
37
The categories of this paper are: supr-con
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio