Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Transformer Based Language Models for Similar Text Retrieval and Ranking   Most approaches for similar text retrieval and ranking with long natural
language queries rely at some level on queries and responses having words in
common with each other. Recent applications of transformer-based neural
language models to text retrieval and ranking problems have been very
promising, but still involve a two-step process in which result candidates are
first obtained through bag-of-words-based approaches, and then reranked by a
neural transformer. In this paper, we introduce novel approaches for
effectively applying neural transformer models to similar text retrieval and
ranking without an initial bag-of-words-based step. By eliminating the
bag-of-words-based step, our approach is able to accurately retrieve and rank
results even when they have no non-stopwords in common with the query. We
accomplish this by using bidirectional encoder representations from
transformers (BERT) to create vectorized representations of sentence-length
texts, along with a vector nearest neighbor search index. We demonstrate both
supervised and unsupervised means of using BERT to accomplish this task.

30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math
4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
21
The categories of this paper are: adap-org
6
The categories of this paper are: astro-ph
27
The categories of this paper are: comp-gas
29
The categories of this paper are: funct-an
31
The categories of this paper are: ao-sci
37
The categories of this paper are: supr-con