Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces
  with Pseudowords   We present a method for exploring regions around individual points in a
contextualized vector space (particularly, BERT space), as a way to investigate
how these regions correspond to word senses. By inducing a contextualized
"pseudoword" as a stand-in for a static embedding in the input layer, and then
performing masked prediction of a word in the sentence, we are able to
investigate the geometry of the BERT-space in a controlled manner around
individual instances. Using our method on a set of carefully constructed
sentences targeting ambiguous English words, we find substantial regularity in
the contextualized space, with regions that correspond to distinct word senses;
but between these regions there are occasionally "sense voids" -- regions that
do not correspond to any intelligible sense.

11
The categories of this paper are: hep-lat
19
The categories of this paper are: eess
26
The categories of this paper are: bayes-an
6
The categories of this paper are: astro-ph
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
36
The categories of this paper are: cmp-lg
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
5
The categories of this paper are: gr-qc
8
The categories of this paper are: hep-ex
16
The categories of this paper are: stat
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
3
The categories of this paper are: physics