  Pre-training and fine-tuning have achieved significant advances in the
information retrieval (IR). A typical approach is to fine-tune all the
parameters of large-scale pre-trained models (PTMs) on downstream tasks. As the
model size and the number of tasks increase greatly, such approach becomes less
feasible and prohibitively expensive. Recently, a variety of
parameter-efficient tuning methods have been proposed in natural language
processing (NLP) that only fine-tune a small number of parameters while still
attaining strong performance. Yet there has been little effort to explore
parameter-efficient tuning for IR.
  In this work, we first conduct a comprehensive study of existing
parameter-efficient tuning methods at both the retrieval and re-ranking stages.
Unlike the promising results in NLP, we find that these methods cannot achieve
comparable performance to full fine-tuning at both stages when updating less
than 1\% of the original model parameters. More importantly, we find that the
existing methods are just parameter-efficient, but not learning-efficient as
they suffer from unstable training and slow convergence. To analyze the
underlying reason, we conduct a theoretical analysis and show that the
separation of the inserted trainable modules makes the optimization difficult.
To alleviate this issue, we propose to inject additional modules alongside the
\acp{PTM} to make the original scattered modules connected. In this way, all
the trainable modules can form a pathway to smooth the loss surface and thus
help stabilize the training process. Experiments at both retrieval and
re-ranking stages show that our method outperforms existing parameter-efficient
methods significantly, and achieves comparable or even better performance over
full fine-tuning.
