Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improving BERT Model Using Contrastive Learning for Biomedical Relation
  Extraction   Contrastive learning has been used to learn a high-quality representation of
the image in computer vision. However, contrastive learning is not widely
utilized in natural language processing due to the lack of a general method of
data augmentation for text data. In this work, we explore the method of
employing contrastive learning to improve the text representation from the BERT
model for relation extraction. The key knob of our framework is a unique
contrastive pre-training step tailored for the relation extraction tasks by
seamlessly integrating linguistic knowledge into the data augmentation.
Furthermore, we investigate how large-scale data constructed from the external
knowledge bases can enhance the generality of contrastive pre-training of BERT.
The experimental results on three relation extraction benchmark datasets
demonstrate that our method can improve the BERT model representation and
achieve state-of-the-art performance. In addition, we explore the
interpretability of models by showing that BERT with contrastive pre-training
relies more on rationales for prediction. Our code and data are publicly
available at: https://github.com/udel-biotm-lab/BERT-CLRE.

6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
12
The categories of this paper are: nucl-th
29
The categories of this paper are: funct-an
11
The categories of this paper are: hep-lat
2
The categories of this paper are: math
19
The categories of this paper are: eess
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
34
The categories of this paper are: chem-ph
24
The categories of this paper are: dg-ga
16
The categories of this paper are: stat
0
The categories of this paper are: hep-ph