Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Long Expressive Memory for Sequence Modeling   We propose a novel method called Long Expressive Memory (LEM) for learning
long-term sequential dependencies. LEM is gradient-based, it can efficiently
process sequential tasks with very long-term dependencies, and it is
sufficiently expressive to be able to learn complicated input-output maps. To
derive LEM, we consider a system of multiscale ordinary differential equations,
as well as a suitable time-discretization of this system. For LEM, we derive
rigorous bounds to show the mitigation of the exploding and vanishing gradients
problem, a well-known challenge for gradient-based recurrent sequential
learning methods. We also prove that LEM can approximate a large class of
dynamical systems to high accuracy. Our empirical results, ranging from image
and time-series classification through dynamical systems prediction to speech
recognition and language modeling, demonstrate that LEM outperforms
state-of-the-art recurrent neural networks, gated recurrent units, and long
short-term memory models.

27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
31
The categories of this paper are: ao-sci
24
The categories of this paper are: dg-ga
35
The categories of this paper are: mtrl-th
5
The categories of this paper are: gr-qc
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
32
The categories of this paper are: atom-ph
17
The categories of this paper are: q-fin
15
The categories of this paper are: nucl-ex