Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Beyond Backprop: Online Alternating Minimization with Auxiliary
  Variables   Despite significant recent advances in deep neural networks, training them
remains a challenge due to the highly non-convex nature of the objective
function. State-of-the-art methods rely on error backpropagation, which suffers
from several well-known issues, such as vanishing and exploding gradients,
inability to handle non-differentiable nonlinearities and to parallelize
weight-updates across layers, and biological implausibility. These limitations
continue to motivate exploration of alternative training algorithms, including
several recently proposed auxiliary-variable methods which break the complex
nested objective function into local subproblems. However, those techniques are
mainly offline (batch), which limits their applicability to extremely large
datasets, as well as to online, continual or reinforcement learning. The main
contribution of our work is a novel online (stochastic/mini-batch) alternating
minimization (AM) approach for training deep neural networks, together with the
first theoretical convergence guarantees for AM in stochastic settings and
promising empirical results on a variety of architectures and datasets.

37
The categories of this paper are: supr-con
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th
10
The categories of this paper are: q-bio
14
The categories of this paper are: math-ph
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
6
The categories of this paper are: astro-ph
15
The categories of this paper are: nucl-ex
17
The categories of this paper are: q-fin