Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Examining Cross-lingual Contextual Embeddings with Orthogonal Structural
  Probes   State-of-the-art contextual embeddings are obtained from large language
models available only for a few languages. For others, we need to learn
representations using a multilingual model. There is an ongoing debate on
whether multilingual embeddings can be aligned in a space shared across many
languages. The novel Orthogonal Structural Probe (Limisiewicz and Mare\v{c}ek,
2021) allows us to answer this question for specific linguistic features and
learn a projection based only on mono-lingual annotated datasets. We evaluate
syntactic (UD) and lexical (WordNet) structural information encoded inmBERT's
contextual representations for nine diverse languages. We observe that for
languages closely related to English, no transformation is needed. The
evaluated information is encoded in a shared cross-lingual embedding space. For
other languages, it is beneficial to apply orthogonal transformation learned
separately for each language. We successfully apply our findings to zero-shot
and few-shot cross-lingual parsing.

12
The categories of this paper are: nucl-th
11
The categories of this paper are: hep-lat
21
The categories of this paper are: adap-org
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
20
The categories of this paper are: acc-phys
36
The categories of this paper are: cmp-lg
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
19
The categories of this paper are: eess
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
0
The categories of this paper are: hep-ph
3
The categories of this paper are: physics