Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Autoregressive Search Engines: Generating Substrings as Document
  Identifiers   Knowledge-intensive language tasks require NLP systems to both provide the
correct answer and retrieve supporting evidence for it in a given corpus.
Autoregressive language models are emerging as the de-facto standard for
generating answers, with newer and more powerful systems emerging at an
astonishing pace. In this paper we argue that all this (and future) progress
can be directly applied to the retrieval problem with minimal intervention to
the models' architecture. Previous work has explored ways to partition the
search space into hierarchical structures and retrieve documents by
autoregressively generating their unique identifier. In this work we propose an
alternative that doesn't force any structure in the search space: using all
ngrams in a passage as its possible identifiers. This setup allows us to use an
autoregressive model to generate and score distinctive ngrams, that are then
mapped to full passages through an efficient data structure. Empirically, we
show this not only outperforms prior autoregressive approaches but also leads
to an average improvement of at least 10 points over more established retrieval
solutions for passage-level retrieval on the KILT benchmark, establishing new
state-of-the-art downstream performance on some datasets, while using a
considerably lighter memory footprint than competing systems. Code and
pre-trained models at https://github.com/facebookresearch/SEAL.

15
The categories of this paper are: nucl-ex
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
14
The categories of this paper are: math-ph
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
29
The categories of this paper are: funct-an
32
The categories of this paper are: atom-ph
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
7
The categories of this paper are: hep-th
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
4
The categories of this paper are: cond-mat
8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph