Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Recurrent Neural Networks with Mixed Hierarchical Structures and EM
  Algorithm for Natural Language Processing   How to obtain hierarchical representations with an increasing level of
abstraction becomes one of the key issues of learning with deep neural
networks. A variety of RNN models have recently been proposed to incorporate
both explicit and implicit hierarchical information in modeling languages in
the literature. In this paper, we propose a novel approach called the latent
indicator layer to identify and learn implicit hierarchical information (e.g.,
phrases), and further develop an EM algorithm to handle the latent indicator
layer in training. The latent indicator layer further simplifies a text's
hierarchical structure, which allows us to seamlessly integrate different
levels of attention mechanisms into the structure. We called the resulting
architecture as the EM-HRNN model. Furthermore, we develop two bootstrap
strategies to effectively and efficiently train the EM-HRNN model on long text
documents. Simulation studies and real data applications demonstrate that the
EM-HRNN model with bootstrap training outperforms other RNN-based models in
document classification tasks. The performance of the EM-HRNN model is
comparable to a Transformer-based method called Bert-base, though the former is
much smaller model and does not require pre-training.

14
The categories of this paper are: math-ph
23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
29
The categories of this paper are: funct-an
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio
21
The categories of this paper are: adap-org
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
27
The categories of this paper are: comp-gas
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
31
The categories of this paper are: ao-sci