Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: LSTMs Compose (and Learn) Bottom-Up   Recent work in NLP shows that LSTM language models capture hierarchical
structure in language data. In contrast to existing work, we consider the
\textit{learning} process that leads to their compositional behavior. For a
closer look at how an LSTM's sequential representations are composed
hierarchically, we present a related measure of Decompositional Interdependence
(DI) between word meanings in an LSTM, based on their gate interactions. We
connect this measure to syntax with experiments on English language data, where
DI is higher on pairs of words with lower syntactic distance. To explore the
inductive biases that cause these compositional representations to arise during
training, we conduct simple experiments on synthetic data. These synthetic
experiments support a specific hypothesis about how hierarchical structures are
discovered over the course of training: that LSTM constituent representations
are learned bottom-up, relying on effective representations of their shorter
children, rather than learning the longer-range relations independently from
children.

32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol
18
The categories of this paper are: econ
9
The categories of this paper are: nlin
4
The categories of this paper are: cond-mat
36
The categories of this paper are: cmp-lg
21
The categories of this paper are: adap-org
0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
6
The categories of this paper are: astro-ph
7
The categories of this paper are: hep-th