Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Hyperlink-induced Pre-training for Passage Retrieval in Open-domain
  Question Answering   To alleviate the data scarcity problem in training question answering
systems, recent works propose additional intermediate pre-training for dense
passage retrieval (DPR). However, there still remains a large discrepancy
between the provided upstream signals and the downstream question-passage
relevance, which leads to less improvement. To bridge this gap, we propose the
HyperLink-induced Pre-training (HLP), a method to pre-train the dense retriever
with the text relevance induced by hyperlink-based topology within Web
documents. We demonstrate that the hyperlink-based structures of dual-link and
co-mention can provide effective relevance signals for large-scale pre-training
that better facilitate downstream passage retrieval. We investigate the
effectiveness of our approach across a wide range of open-domain QA datasets
under zero-shot, few-shot, multi-hop, and out-of-domain scenarios. The
experiments show our HLP outperforms the BM25 by up to 7 points as well as
other pre-training methods by more than 10 points in terms of top-20 retrieval
accuracy under the zero-shot scenario. Furthermore, HLP significantly
outperforms other pre-training methods under the other scenarios.

14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
8
The categories of this paper are: hep-ex
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
3
The categories of this paper are: physics
36
The categories of this paper are: cmp-lg
16
The categories of this paper are: stat
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an
9
The categories of this paper are: nlin
2
The categories of this paper are: math
28
The categories of this paper are: alg-geom
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int