  Multiple hypothesis testing, a situation when we wish to consider many
hypotheses, is a core problem in statistical inference that arises in almost
every scientific field. In this setting, controlling the false discovery rate
(FDR), which is the expected proportion of type I error, is an important
challenge for making meaningful inferences. In this paper, we consider the
problem of controlling FDR in an online manner. Concretely, we consider an
ordered, possibly infinite, sequence of hypotheses, arriving one at each
timestep, and for each hypothesis we observe a p-value along with a set of
features specific to that hypothesis. The decision whether or not to reject the
current hypothesis must be made immediately at each timestep, before the next
hypothesis is observed. The model of multi-dimensional feature set provides a
very general way of leveraging the auxiliary information in the data which
helps in maximizing the number of discoveries.
  We propose a new class of powerful online testing procedures, where the
rejections thresholds (significance levels) are learnt sequentially by
incorporating contextual information and previous results. We prove that any
rule in this class controls online FDR under some standard assumptions. We then
focus on a subclass of these procedures, based on weighting significance
levels, to derive a practical algorithm that learns a parametric weight
function in an online fashion to gain more discoveries. We also theoretically
prove, in a stylized setting, that our proposed procedures would lead to an
increase in the achieved statistical power over a popular online testing
procedure proposed by Javanmard & Montanari (2018). Finally, we demonstrate the
favorable performance of our procedure, by comparing it to state-of-the-art
online multiple testing procedures, on both synthetic data and real data
generated from different applications.
