Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: K-XLNet: A General Method for Combining Explicit Knowledge with Language
  Model Pretraining   Though pre-trained language models such as Bert and XLNet, have rapidly
advanced the state-of-the-art on many NLP tasks, they implicit semantics only
relying on surface information between words in corpus. Intuitively, background
knowledge influences the efficacy of understanding. Inspired by this common
sense, we focus on improving model pretraining by leveraging explicit
knowledge. Different from recent research that optimize pretraining model by
knowledge masking strategies, we propose a simple but general method to combine
explicit knowledge with pretraining. To be specific, we first match knowledge
facts from knowledge graph (KG) and then add a knowledge injunction layer to
transformer directly without changing its architecture. The present study seeks
to find the direct impact of explicit knowledge on transformer per-training. We
conduct experiments on various datasets for different downstream tasks. The
experimental results show that solely by adding external knowledge to
transformer can improve the learning performance on many NLP tasks.

9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
35
The categories of this paper are: mtrl-th
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
31
The categories of this paper are: ao-sci
37
The categories of this paper are: supr-con