Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Path-Normalized Optimization of Recurrent Neural Networks with ReLU
  Activations   We investigate the parameter-space geometry of recurrent neural networks
(RNNs), and develop an adaptation of path-SGD optimization method, attuned to
this geometry, that can learn plain RNNs with ReLU activations. On several
datasets that require capturing long-term dependency structure, we show that
path-SGD can significantly improve trainability of ReLU RNNs compared to RNNs
trained with SGD, even with various recently suggested initialization schemes.

15
The categories of this paper are: nucl-ex
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
19
The categories of this paper are: eess
21
The categories of this paper are: adap-org
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
26
The categories of this paper are: bayes-an
2
The categories of this paper are: math
12
The categories of this paper are: nucl-th
17
The categories of this paper are: q-fin