Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: State-Regularized Recurrent Neural Networks   Recurrent neural networks are a widely used class of neural architectures.
They have, however, two shortcomings. First, it is difficult to understand what
exactly they learn. Second, they tend to work poorly on sequences requiring
long-term memorization, despite having this capacity in principle. We aim to
address both shortcomings with a class of recurrent networks that use a
stochastic state transition mechanism between cell applications. This
mechanism, which we term state-regularization, makes RNNs transition between a
finite set of learnable states. We evaluate state-regularized RNNs on (1)
regular languages for the purpose of automata extraction; (2) nonregular
languages such as balanced parentheses, palindromes, and the copy task where
external memory is required; and (3) real-word sequence learning tasks for
sentiment analysis, visual object recognition, and language modeling. We show
that state-regularization (a) simplifies the extraction of finite state
automata modeling an RNN's state transition dynamics; (b) forces RNNs to
operate more like automata with external memory and less like finite state
machines; (c) makes RNNs have better interpretability and explainability.

10
The categories of this paper are: q-bio
6
The categories of this paper are: astro-ph
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg
35
The categories of this paper are: mtrl-th
9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
2
The categories of this paper are: math
7
The categories of this paper are: hep-th