Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Circles are like Ellipses, or Ellipses are like Circles? Measuring the
  Degree of Asymmetry of Static and Contextual Embeddings and the Implications
  to Representation Learning   Human judgments of word similarity have been a popular method of evaluating
the quality of word embedding. But it fails to measure the geometry properties
such as asymmetry. For example, it is more natural to say "Ellipses are like
Circles" than "Circles are like Ellipses". Such asymmetry has been observed
from a psychoanalysis test called word evocation experiment, where one word is
used to recall another. Although useful, such experimental data have been
significantly understudied for measuring embedding quality. In this paper, we
use three well-known evocation datasets to gain insights into asymmetry
encoding of embedding. We study both static embedding as well as contextual
embedding, such as BERT. Evaluating asymmetry for BERT is generally hard due to
the dynamic nature of embedding. Thus, we probe BERT's conditional
probabilities (as a language model) using a large number of Wikipedia contexts
to derive a theoretically justifiable Bayesian asymmetry score. The result
shows that contextual embedding shows randomness than static embedding on
similarity judgments while performing well on asymmetry judgment, which aligns
with its strong performance on "extrinsic evaluations" such as text
classification. The asymmetry judgment and the Bayesian approach provides a new
perspective to evaluate contextual embedding on intrinsic evaluation, and its
comparison to similarity evaluation concludes our work with a discussion on the
current state and the future of representation learning.

30
The categories of this paper are: q-alg
2
The categories of this paper are: math
7
The categories of this paper are: hep-th
24
The categories of this paper are: dg-ga
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
26
The categories of this paper are: bayes-an
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
15
The categories of this paper are: nucl-ex
16
The categories of this paper are: stat
6
The categories of this paper are: astro-ph
8
The categories of this paper are: hep-ex
37
The categories of this paper are: supr-con
11
The categories of this paper are: hep-lat