  Studies accumulate over time and meta-analyses are mainly retrospective.
These two characteristics introduce dependencies between the analysis time, at
which a series of studies is up for meta-analysis, and results within the
series. Dependencies introduce bias --- Accumulation Bias --- and invalidate
the sampling distribution assumed for p-value tests, thus inflating type-I
errors. But dependencies are also inevitable, since for science to accumulate
efficiently, new research needs to be informed by past results. Here, we
investigate various ways in which time influences error control in
meta-analysis testing. We introduce an Accumulation Bias Framework that allows
us to model a wide variety of practically occurring dependencies, including
study series accumulation, meta-analysis timing, and approaches to multiple
testing in living systematic reviews. The strength of this framework is that it
shows how all dependencies affect p-value-based tests in a similar manner. This
leads to two main conclusions. First, Accumulation Bias is inevitable, and even
if it can be approximated and accounted for, no valid p-value tests can be
constructed. Second, tests based on likelihood ratios withstand Accumulation
Bias: they provide bounds on error probabilities that remain valid despite the
bias. We leave the reader with a choice between two proposals to consider time
in error control: either treat individual (primary) studies and meta-analyses
as two separate worlds --- each with their own timing --- or integrate
individual studies in the meta-analysis world. Taking up likelihood ratios in
either approach allows for valid tests that relate well to the accumulating
nature of scientific knowledge. Likelihood ratios can be interpreted as betting
profits, earned in previous studies and invested in new ones, while the
meta-analyst is allowed to cash out at any time and advise against future
studies.
