Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Model Selection for Cross-Lingual Transfer   Transformers that are pre-trained on multilingual corpora, such as, mBERT and
XLM-RoBERTa, have achieved impressive cross-lingual transfer capabilities. In
the zero-shot transfer setting, only English training data is used, and the
fine-tuned model is evaluated on another target language. While this works
surprisingly well, substantial variance has been observed in target language
performance between different fine-tuning runs, and in the zero-shot setup, no
target-language development data is available to select among multiple
fine-tuned models. Prior work has relied on English dev data to select among
models that are fine-tuned with different learning rates, number of steps and
other hyperparameters, often resulting in suboptimal choices. In this paper, we
show that it is possible to select consistently better models when small
amounts of annotated data are available in auxiliary pivot languages. We
propose a machine learning approach to model selection that uses the fine-tuned
model's own internal representations to predict its cross-lingual capabilities.
In extensive experiments we find that this method consistently selects better
models than English validation data across twenty five languages (including
eight low-resource languages), and often achieves results that are comparable
to model selection using target language development data.

4
The categories of this paper are: cond-mat
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
21
The categories of this paper are: adap-org
8
The categories of this paper are: hep-ex
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
31
The categories of this paper are: ao-sci
25
The categories of this paper are: patt-sol
23
The categories of this paper are: solv-int
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
2
The categories of this paper are: math
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con