Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT   We present a novel way of injecting factual knowledge about entities into the
pretrained BERT model (Devlin et al., 2019): We align Wikipedia2Vec entity
vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use
the aligned entity vectors as if they were wordpiece vectors. The resulting
entity-enhanced version of BERT (called E-BERT) is similar in spirit to ERNIE
(Zhang et al., 2019) and KnowBert (Peters et al., 2019), but it requires no
expensive further pretraining of the BERT encoder. We evaluate E-BERT on
unsupervised question answering (QA), supervised relation classification (RC)
and entity linking (EL). On all three tasks, E-BERT outperforms BERT and other
baselines. We also show quantitatively that the original BERT model is overly
reliant on the surface form of entity names (e.g., guessing that someone with
an Italian-sounding name speaks Italian), and that E-BERT mitigates this
problem.

21
The categories of this paper are: adap-org
20
The categories of this paper are: acc-phys
29
The categories of this paper are: funct-an
37
The categories of this paper are: supr-con
28
The categories of this paper are: alg-geom
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
13
The categories of this paper are: quant-ph
6
The categories of this paper are: astro-ph
15
The categories of this paper are: nucl-ex
36
The categories of this paper are: cmp-lg