Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Three-level Hierarchical Transformer Networks for Long-sequence and
  Multiple Clinical Documents Classification   We present a Three-level Hierarchical Transformer Network (3-level-HTN) for
modeling long-term dependencies across clinical notes for the purpose of
patient-level prediction. The network is equipped with three levels of
Transformer-based encoders to learn progressively from words to sentences,
sentences to notes, and finally notes to patients. The first level from word to
sentence directly applies a pre-trained BERT model as a fully trainable
component. While the second and third levels both implement a stack of
transformer-based encoders, before the final patient representation is fed into
a classification layer for clinical predictions. Compared to conventional BERT
models, our model increases the maximum input length from 512 tokens to much
longer sequences that are appropriate for modeling large numbers of clinical
notes. We empirically examine different hyper-parameters to identify an optimal
trade-off given computational resource limits. Our experiment results on the
MIMIC-III dataset for different prediction tasks demonstrate that the proposed
Hierarchical Transformer Network outperforms previous state-of-the-art models,
including but not limited to BigBird.

14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
15
The categories of this paper are: nucl-ex
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
10
The categories of this paper are: q-bio
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
6
The categories of this paper are: astro-ph
35
The categories of this paper are: mtrl-th
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
19
The categories of this paper are: eess