Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative
  Study   Unsupervised pretraining is an integral part of many natural language
processing systems, and transfer learning with language models has achieved
remarkable results in many downstream tasks. In the clinical application of
medical code assignment, diagnosis and procedure codes are inferred from
lengthy clinical notes such as hospital discharge summaries. However, it is not
clear if pretrained models are useful for medical code prediction without
further architecture engineering. This paper conducts a comprehensive
quantitative analysis of various contextualized language models' performance,
pretrained in different domains, for medical code assignment from clinical
notes. We propose a hierarchical fine-tuning architecture to capture
interactions between distant words and adopt label-wise attention to exploit
label information. Contrary to current trends, we demonstrate that a carefully
trained classical CNN outperforms attention-based models on a MIMIC-III subset
with frequent codes. Our empirical findings suggest directions for improving
the medical code assignment application.

13
The categories of this paper are: quant-ph
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph
16
The categories of this paper are: stat
28
The categories of this paper are: alg-geom
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
23
The categories of this paper are: solv-int
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
15
The categories of this paper are: nucl-ex
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th