Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Training Neural Networks Without Gradients: A Scalable ADMM Approach   With the growing importance of large network models and enormous training
datasets, GPUs have become increasingly necessary to train neural networks.
This is largely because conventional optimization algorithms rely on stochastic
gradient methods that don't scale well to large numbers of cores in a cluster
setting. Furthermore, the convergence of all gradient methods, including batch
methods, suffers from common problems like saturation effects, poor
conditioning, and saddle points. This paper explores an unconventional training
method that uses alternating direction methods and Bregman iteration to train
networks without gradient descent steps. The proposed method reduces the
network training problem to a sequence of minimization sub-steps that can each
be solved globally in closed form. The proposed method is advantageous because
it avoids many of the caveats that make gradient methods slow on highly
non-convex problems. The method exhibits strong scaling in the distributed
setting, yielding linear speedups even when split over thousands of cores.

26
The categories of this paper are: bayes-an
2
The categories of this paper are: math
13
The categories of this paper are: quant-ph
19
The categories of this paper are: eess
23
The categories of this paper are: solv-int
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
10
The categories of this paper are: q-bio
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas