Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: El Departamento de Nosotros: How Machine Translated Corpora Affects
  Language Models in MRC Tasks   Pre-training large-scale language models (LMs) requires huge amounts of text
corpora. LMs for English enjoy ever growing corpora of diverse language
resources. However, less resourced languages and their mono- and multilingual
LMs often struggle to obtain bigger datasets. A typical approach in this case
implies using machine translation of English corpora to a target language. In
this work, we study the caveats of applying directly translated corpora for
fine-tuning LMs for downstream natural language processing tasks and
demonstrate that careful curation along with post-processing lead to improved
performance and overall LMs robustness. In the empirical evaluation, we perform
a comparison of directly translated against curated Spanish SQuAD datasets on
both user and system levels. Further experimental results on XQuAD and MLQA
transfer-learning evaluation question answering tasks show that presumably
multilingual LMs exhibit more resilience to machine translation artifacts in
terms of the exact match score.

2
The categories of this paper are: math
11
The categories of this paper are: hep-lat
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
14
The categories of this paper are: math-ph
9
The categories of this paper are: nlin
16
The categories of this paper are: stat
31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th