Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Pre-training of Graph Augmented Transformers for Medication
  Recommendation   Medication recommendation is an important healthcare application. It is
commonly formulated as a temporal prediction task. Hence, most existing works
only utilize longitudinal electronic health records (EHRs) from a small number
of patients with multiple visits ignoring a large number of patients with a
single visit (selection bias). Moreover, important hierarchical knowledge such
as diagnosis hierarchy is not leveraged in the representation learning process.
To address these challenges, we propose G-BERT, a new model to combine the
power of Graph Neural Networks (GNNs) and BERT (Bidirectional Encoder
Representations from Transformers) for medical code representation and
medication recommendation. We use GNNs to represent the internal hierarchical
structures of medical codes. Then we integrate the GNN representation into a
transformer-based visit encoder and pre-train it on EHR data from patients only
with a single visit. The pre-trained visit encoder and representation are then
fine-tuned for downstream predictive tasks on longitudinal EHRs from patients
with multiple visits. G-BERT is the first to bring the language model
pre-training schema into the healthcare domain and it achieved state-of-the-art
performance on the medication recommendation task.

13
The categories of this paper are: quant-ph
16
The categories of this paper are: stat
20
The categories of this paper are: acc-phys
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
6
The categories of this paper are: astro-ph