Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Training Recurrent Neural Networks by Diffusion   This work presents a new algorithm for training recurrent neural networks
(although ideas are applicable to feedforward networks as well). The algorithm
is derived from a theory in nonconvex optimization related to the diffusion
equation. The contributions made in this work are two fold. First, we show how
some seemingly disconnected mechanisms used in deep learning such as smart
initialization, annealed learning rate, layerwise pretraining, and noise
injection (as done in dropout and SGD) arise naturally and automatically from
this framework, without manually crafting them into the algorithms. Second, we
present some preliminary results on comparing the proposed method against SGD.
It turns out that the new algorithm can achieve similar level of generalization
accuracy of SGD in much fewer number of epochs.

0
The categories of this paper are: hep-ph
5
The categories of this paper are: gr-qc
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
33
The categories of this paper are: plasm-ph
22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ
32
The categories of this paper are: atom-ph