Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Cross-Lingual Ability of Multilingual Masked Language Models: A Study of
  Language Structure   Multilingual pre-trained language models, such as mBERT and XLM-R, have shown
impressive cross-lingual ability. Surprisingly, both of them use multilingual
masked language model (MLM) without any cross-lingual supervision or aligned
data. Despite the encouraging results, we still lack a clear understanding of
why cross-lingual ability could emerge from multilingual MLM. In our work, we
argue that cross-language ability comes from the commonality between languages.
Specifically, we study three language properties: constituent order,
composition and word co-occurrence. First, we create an artificial language by
modifying property in source language. Then we study the contribution of
modified property through the change of cross-language transfer results on
target language. We conduct experiments on six languages and two cross-lingual
NLP tasks (textual entailment, sentence retrieval). Our main conclusion is that
the contribution of constituent order and word co-occurrence is limited, while
the composition is more crucial to the success of cross-linguistic transfer.

24
The categories of this paper are: dg-ga
5
The categories of this paper are: gr-qc
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
0
The categories of this paper are: hep-ph
31
The categories of this paper are: ao-sci