Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Parsing with Pretrained Language Models, Multiple Datasets, and Dataset
  Embeddings   With an increase of dataset availability, the potential for learning from a
variety of data sources has increased. One particular method to improve
learning from multiple data sources is to embed the data source during
training. This allows the model to learn generalizable features as well as
distinguishing features between datasets. However, these dataset embeddings
have mostly been used before contextualized transformer-based embeddings were
introduced in the field of Natural Language Processing. In this work, we
compare two methods to embed datasets in a transformer-based multilingual
dependency parser, and perform an extensive evaluation. We show that: 1)
embedding the dataset is still beneficial with these models 2) performance
increases are highest when embedding the dataset at the encoder level 3)
unsurprisingly, we confirm that performance increases are highest for small
datasets and datasets with a low baseline score. 4) we show that training on
the combination of all datasets performs similarly to designing smaller
clusters based on language-relatedness.

36
The categories of this paper are: cmp-lg
29
The categories of this paper are: funct-an
20
The categories of this paper are: acc-phys
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
8
The categories of this paper are: hep-ex
17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
35
The categories of this paper are: mtrl-th
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
3
The categories of this paper are: physics
18
The categories of this paper are: econ