  Ranking is at the core of many artificial intelligence (AI) applications,
including search engines, recommender systems, etc. Modern ranking systems are
often constructed with learning-to-rank (LTR) models built from user behavior
signals. While previous studies have demonstrated the effectiveness of using
user behavior signals (e.g., clicks) as both features and labels of LTR
algorithms, we argue that existing LTR algorithms that indiscriminately treat
behavior and non-behavior signals in input features could lead to suboptimal
performance in practice. Particularly because user behavior signals often have
strong correlations with the ranking objective and can only be collected on
items that have already been shown to users, directly using behavior signals in
LTR could create an exploitation bias that hurts the system performance in the
long run.
  To address the exploitation bias, we propose EBRank, an empirical Bayes-based
uncertainty-aware ranking algorithm. Specifically, to overcome exploitation
bias brought by behavior features in ranking models, EBRank uses a sole
non-behavior feature based prior model to get a prior estimation of relevance.
In the dynamic training and serving of ranking systems, EBRank uses the
observed user behaviors to update posterior relevance estimation instead of
concatenating behaviors as features in ranking models. Besides, EBRank
additionally applies an uncertainty-aware exploration strategy to explore
actively, collect user behaviors for empirical Bayesian modeling and improve
ranking performance. Experiments on three public datasets show that EBRank is
effective, practical and significantly outperforms state-of-the-art ranking
algorithms.
