  State-of-the-art neural (re)rankers are notoriously data-hungry which --
given the lack of large-scale training data in languages other than English --
makes them rarely used in multilingual and cross-lingual retrieval settings.
Current approaches therefore commonly transfer rankers trained on English data
to other languages and cross-lingual setups by means of multilingual encoders:
they fine-tune all parameters of pretrained massively multilingual Transformers
(MMTs, e.g., multilingual BERT) on English relevance judgments, and then deploy
them in the target language(s). In this work, we show that two
parameter-efficient approaches to cross-lingual transfer, namely Sparse
Fine-Tuning Masks (SFTMs) and Adapters, allow for a more lightweight and more
effective zero-shot transfer to multilingual and cross-lingual retrieval tasks.
We first train language adapters (or SFTMs) via Masked Language Modelling and
then train retrieval (i.e., reranking) adapters (SFTMs) on top, while keeping
all other parameters fixed. At inference, this modular design allows us to
compose the ranker by applying the (re)ranking adapter (or SFTM) trained with
source language data together with the language adapter (or SFTM) of a target
language. We carry out a large scale evaluation on the CLEF-2003 and HC4
benchmarks and additionally, as another contribution, extend the former with
queries in three new languages: Kyrgyz, Uyghur and Turkish. The proposed
parameter-efficient methods outperform standard zero-shot transfer with full
MMT fine-tuning, while being more modular and reducing training times. The
gains are particularly pronounced for low-resource languages, where our
approaches also substantially outperform the competitive machine
translation-based rankers.
