Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: AT-BERT: Adversarial Training BERT for Acronym Identification Winning
  Solution for SDU@AAAI-21   Acronym identification focuses on finding the acronyms and the phrases that
have been abbreviated, which is crucial for scientific document understanding
tasks. However, the limited size of manually annotated datasets hinders further
improvement for the problem. Recent breakthroughs of language models
pre-trained on large corpora clearly show that unsupervised pre-training can
vastly improve the performance of downstream tasks. In this paper, we present
an Adversarial Training BERT method named AT-BERT, our winning solution to
acronym identification task for Scientific Document Understanding (SDU)
Challenge of AAAI 2021. Specifically, the pre-trained BERT is adopted to
capture better semantic representation. Then we incorporate the FGM adversarial
training strategy into the fine-tuning of BERT, which makes the model more
robust and generalized. Furthermore, an ensemble mechanism is devised to
involve the representations learned from multiple BERT variants. Assembling all
these components together, the experimental results on the SciAI dataset show
that our proposed approach outperforms all other competitive state-of-the-art
methods.

25
The categories of this paper are: patt-sol
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
36
The categories of this paper are: cmp-lg
26
The categories of this paper are: bayes-an
30
The categories of this paper are: q-alg
2
The categories of this paper are: math
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
29
The categories of this paper are: funct-an
28
The categories of this paper are: alg-geom
10
The categories of this paper are: q-bio
9
The categories of this paper are: nlin
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph