Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Transformer-based Language Model Fine-tuning Methods for COVID-19 Fake
  News Detection   With the pandemic of COVID-19, relevant fake news is spreading all over the
sky throughout the social media. Believing in them without discrimination can
cause great trouble to people's life. However, universal language models may
perform weakly in these fake news detection for lack of large-scale annotated
data and sufficient semantic understanding of domain-specific knowledge. While
the model trained on corresponding corpora is also mediocre for insufficient
learning. In this paper, we propose a novel transformer-based language model
fine-tuning approach for these fake news detection. First, the token vocabulary
of individual model is expanded for the actual semantics of professional
phrases. Second, we adapt the heated-up softmax loss to distinguish the
hard-mining samples, which are common for fake news because of the
disambiguation of short text. Then, we involve adversarial training to improve
the model's robustness. Last, the predicted features extracted by universal
language model RoBERTa and domain-specific model CT-BERT are fused by one
multiple layer perception to integrate fine-grained and high-level specific
representations. Quantitative experimental results evaluated on existing
COVID-19 fake news dataset show its superior performances compared to the
state-of-the-art methods among various evaluation metrics. Furthermore, the
best weighted average F1 score achieves 99.02%.

10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
35
The categories of this paper are: mtrl-th
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
2
The categories of this paper are: math
33
The categories of this paper are: plasm-ph
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
8
The categories of this paper are: hep-ex
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
9
The categories of this paper are: nlin
4
The categories of this paper are: cond-mat
5
The categories of this paper are: gr-qc