Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Various Length Dependence by Dual Recurrent Neural Networks   Recurrent neural networks (RNNs) are widely used as a memory model for
sequence-related problems. Many variants of RNN have been proposed to solve the
gradient problems of training RNNs and process long sequences. Although some
classical models have been proposed, capturing long-term dependence while
responding to short-term changes remains a challenge. To this problem, we
propose a new model named Dual Recurrent Neural Networks (DuRNN). The DuRNN
consists of two parts to learn the short-term dependence and progressively
learn the long-term dependence. The first part is a recurrent neural network
with constrained full recurrent connections to deal with short-term dependence
in sequence and generate short-term memory. Another part is a recurrent neural
network with independent recurrent connections which helps to learn long-term
dependence and generate long-term memory. A selection mechanism is added
between two parts to help the needed long-term information transfer to the
independent neurons. Multiple modules can be stacked to form a multi-layer
model for better performance. Our contributions are: 1) a new recurrent model
developed based on the divide-and-conquer strategy to learn long and short-term
dependence separately, and 2) a selection mechanism to enhance the separating
and learning of different temporal scales of dependence. Both theoretical
analysis and extensive experiments are conducted to validate the performance of
our model, and we also conduct simple visualization experiments and ablation
analyses for the model interpretability. Experimental results indicate that the
proposed DuRNN model can handle not only very long sequences (over 5000 time
steps), but also short sequences very well. Compared with many state-of-the-art
RNN models, our model has demonstrated efficient and better performance.

25
The categories of this paper are: patt-sol
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
21
The categories of this paper are: adap-org
13
The categories of this paper are: quant-ph
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
35
The categories of this paper are: mtrl-th
10
The categories of this paper are: q-bio
37
The categories of this paper are: supr-con
34
The categories of this paper are: chem-ph