Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Evaluating Transferability of BERT Models on Uralic Languages   Transformer-based language models such as BERT have outperformed previous
models on a large number of English benchmarks, but their evaluation is often
limited to English or a small number of well-resourced languages. In this work,
we evaluate monolingual, multilingual, and randomly initialized language models
from the BERT family on a variety of Uralic languages including Estonian,
Finnish, Hungarian, Erzya, Moksha, Karelian, Livvi, Komi Permyak, Komi Zyrian,
Northern S\'ami, and Skolt S\'ami. When monolingual models are available
(currently only et, fi, hu), these perform better on their native language, but
in general they transfer worse than multilingual models or models of
genetically unrelated languages that share the same character set. Remarkably,
straightforward transfer of high-resource models, even without special efforts
toward hyperparameter optimization, yields what appear to be state of the art
POS and NER tools for the minority Uralic languages where there is sufficient
data for finetuning.

34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
29
The categories of this paper are: funct-an
25
The categories of this paper are: patt-sol