Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unsupervised Attention-based Sentence-Level Meta-Embeddings from
  Contextualised Language Models   A variety of contextualised language models have been proposed in the NLP
community, which are trained on diverse corpora to produce numerous Neural
Language Models (NLMs). However, different NLMs have reported different levels
of performances in downstream NLP applications when used as text
representations. We propose a sentence-level meta-embedding learning method
that takes independently trained contextualised word embedding models and
learns a sentence embedding that preserves the complementary strengths of the
input source NLMs. Our proposed method is unsupervised and is not tied to a
particular downstream task, which makes the learnt meta-embeddings in principle
applicable to different tasks that require sentence representations.
Specifically, we first project the token-level embeddings obtained by the
individual NLMs and learn attention weights that indicate the contributions of
source embeddings towards their token-level meta-embeddings. Next, we apply
mean and max pooling to produce sentence-level meta-embeddings from token-level
meta-embeddings. Experimental results on semantic textual similarity benchmarks
show that our proposed unsupervised sentence-level meta-embedding method
outperforms previously proposed sentence-level meta-embedding methods as well
as a supervised baseline.

35
The categories of this paper are: mtrl-th
23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
36
The categories of this paper are: cmp-lg
32
The categories of this paper are: atom-ph
26
The categories of this paper are: bayes-an
7
The categories of this paper are: hep-th
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
8
The categories of this paper are: hep-ex
2
The categories of this paper are: math
34
The categories of this paper are: chem-ph
37
The categories of this paper are: supr-con
3
The categories of this paper are: physics
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga