Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Prototypical Recurrent Unit   Despite the great successes of deep learning, the effectiveness of deep
neural networks has not been understood at any theoretical depth. This work is
motivated by the thrust of developing a deeper understanding of recurrent
neural networks, particularly LSTM/GRU-like networks. As the highly complex
structure of the recurrent unit in LSTM and GRU networks makes them difficult
to analyze, our methodology in this research theme is to construct an
alternative recurrent unit that is as simple as possible and yet also captures
the key components of LSTM/GRU recurrent units. Such a unit can then be used
for the study of recurrent networks and its structural simplicity may allow
easier analysis. Towards that goal, we take a system-theoretic perspective to
design a new recurrent unit, which we call the prototypical recurrent unit
(PRU). Not only having minimal complexity, PRU is demonstrated experimentally
to have comparable performance to GRU and LSTM unit. This establishes PRU
networks as a prototype for future study of LSTM/GRU-like recurrent networks.
This paper also studies the memorization abilities of LSTM, GRU and PRU
networks, motivated by the folk belief that such networks possess long-term
memory. For this purpose, we design a simple and controllable task, called
``memorization problem'', where the networks are trained to memorize certain
targeted information. We show that the memorization performance of all three
networks depends on the amount of targeted information, the amount of
``interfering" information, and the state space dimension of the recurrent
unit. Experiments are also performed for another controllable task, the adding
problem, and similar conclusions are obtained.

17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
16
The categories of this paper are: stat
4
The categories of this paper are: cond-mat
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
36
The categories of this paper are: cmp-lg
32
The categories of this paper are: atom-ph
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
7
The categories of this paper are: hep-th
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol