Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Optimal Kronecker-Sum Approximation of Real Time Recurrent Learning   One of the central goals of Recurrent Neural Networks (RNNs) is to learn
long-term dependencies in sequential data. Nevertheless, the most popular
training method, Truncated Backpropagation through Time (TBPTT), categorically
forbids learning dependencies beyond the truncation horizon. In contrast, the
online training algorithm Real Time Recurrent Learning (RTRL) provides
untruncated gradients, with the disadvantage of impractically large
computational costs. Recently published approaches reduce these costs by
providing noisy approximations of RTRL. We present a new approximation
algorithm of RTRL, Optimal Kronecker-Sum Approximation (OK). We prove that OK
is optimal for a class of approximations of RTRL, which includes all approaches
published so far. Additionally, we show that OK has empirically negligible
noise: Unlike previous algorithms it matches TBPTT in a real world task
(character-level Penn TreeBank) and can exploit online parameter updates to
outperform TBPTT in a synthetic string memorization task. Code availiable on
github.

11
The categories of this paper are: hep-lat
8
The categories of this paper are: hep-ex
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
35
The categories of this paper are: mtrl-th
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat