Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent
  Networks   A major drawback of backpropagation through time (BPTT) is the difficulty of
learning long-term dependencies, coming from having to propagate credit
information backwards through every single step of the forward computation.
This makes BPTT both computationally impractical and biologically implausible.
For this reason, full backpropagation through time is rarely used on long
sequences, and truncated backpropagation through time is used as a heuristic.
However, this usually leads to biased estimates of the gradient in which longer
term dependencies are ignored. Addressing this issue, we propose an alternative
algorithm, Sparse Attentive Backtracking, which might also be related to
principles used by brains to learn long-term dependencies. Sparse Attentive
Backtracking learns an attention mechanism over the hidden states of the past
and selectively backpropagates through paths with high attention weights. This
allows the model to learn long term dependencies while only backtracking for a
small number of time steps, not just from the recent past but also from
attended relevant past states.

17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
6
The categories of this paper are: astro-ph
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
18
The categories of this paper are: econ
22
The categories of this paper are: chao-dyn
2
The categories of this paper are: math
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys