  We present a novel approach to causal measurement for advertising, namely to
use exogenous variation in advertising exposure (RCTs) for a subset of ad
campaigns to build a model that can predict the causal effect of ad campaigns
that were run without RCTs. This approach -- Predictive Incrementality by
Experimentation (PIE) -- frames the task of estimating the causal effect of an
ad campaign as a prediction problem, with the unit of observation being an RCT
itself. In contrast, traditional causal inference approaches with observational
data seek to adjust covariate imbalance at the user level. A key insight is to
use post-campaign features, such as last-click conversion counts, that do not
require an RCT, as features in our predictive model. We find that our PIE model
recovers RCT-derived incremental conversions per dollar (ICPD) much better than
the program evaluation approaches analyzed in Gordon et al. (forthcoming). The
prediction errors from the best PIE model are 48%, 42%, and 62% of the
RCT-based average ICPD for upper-, mid-, and lower-funnel conversion outcomes,
respectively. In contrast, across the same data, the average prediction error
of stratified propensity score matching exceeds 491%, and that of
double/debiased machine learning exceeds 2,904%. Using a decision-making
framework inspired by industry, we show that PIE leads to different decisions
compared to RCTs for only 6% of upper-funnel, 7% of mid-funnel, and 13% of
lower-funnel outcomes. We conclude that PIE could enable advertising platforms
to scale causal ad measurement by extrapolating from a limited number of RCTs
to a large set of non-experimental ad campaigns.
