Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multi-Sense Language Modelling   The effectiveness of a language model is influenced by its token
representations, which must encode contextual information and handle the same
word form having a plurality of meanings (polysemy). Currently, none of the
common language modelling architectures explicitly model polysemy. We propose a
language model which not only predicts the next word, but also its sense in
context. We argue that this higher prediction granularity may be useful for end
tasks such as assistive writing, and allow for more a precise linking of
language models with knowledge bases. We find that multi-sense language
modelling requires architectures that go beyond standard language models, and
here propose a structured prediction framework that decomposes the task into a
word followed by a sense prediction task. To aid sense prediction, we utilise a
Graph Attention Network, which encodes definitions and example uses of word
senses. Overall, we find that multi-sense language modelling is a highly
challenging task, and suggest that future work focus on the creation of more
annotated training datasets.

22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
10
The categories of this paper are: q-bio
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
14
The categories of this paper are: math-ph
12
The categories of this paper are: nucl-th
7
The categories of this paper are: hep-th