Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improving Transformer-Kernel Ranking Model Using Conformer and Query
  Term Independence   The Transformer-Kernel (TK) model has demonstrated strong reranking
performance on the TREC Deep Learning benchmark -- and can be considered to be
an efficient (but slightly less effective) alternative to other
Transformer-based architectures that employ (i) large-scale pretraining (high
training cost), (ii) joint encoding of query and document (high inference
cost), and (iii) larger number of Transformer layers (both high training and
high inference costs). Since, a variant of the TK model -- called TKL -- has
been developed that incorporates local self-attention to efficiently process
longer input sequences in the context of document ranking. In this work, we
propose a novel Conformer layer as an alternative approach to scale TK to
longer input sequences. Furthermore, we incorporate query term independence and
explicit term matching to extend the model to the full retrieval setting. We
benchmark our models under the strictly blind evaluation setting of the TREC
2020 Deep Learning track and find that our proposed architecture changes lead
to improved retrieval quality over TKL. Our best model also outperforms all
non-neural runs ("trad") and two-thirds of the pretrained Transformer-based
runs ("nnlm") on NDCG@10.

5
The categories of this paper are: gr-qc
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
20
The categories of this paper are: acc-phys
16
The categories of this paper are: stat
7
The categories of this paper are: hep-th
25
The categories of this paper are: patt-sol
37
The categories of this paper are: supr-con
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
8
The categories of this paper are: hep-ex
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
11
The categories of this paper are: hep-lat