Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Method for Measuring Network Effects of One-to-One Communication
  Features in Online A/B Tests   A/B testing is an important decision making tool in product development
because can provide an accurate estimate of the average treatment effect of a
new features, which allows developers to understand how the business impact of
new changes to products or algorithms. However, an important assumption of A/B
testing, Stable Unit Treatment Value Assumption (SUTVA), is not always a valid
assumption to make, especially for products that facilitate interactions
between individuals. In contexts like one-to-one messaging we should expect
network interference; if an experimental manipulation is effective, behavior of
the treatment group is likely to influence members in the control group by
sending them messages, violating this assumption. In this paper, we propose a
novel method that can be used to account for network effects when A/B testing
changes to one-to-one interactions. Our method is an edge-based analysis that
can be applied to standard Bernoulli randomized experiments to retrieve an
average treatment effect that is not influenced by network interference. We
develop a theoretical model, and methods for computing point estimates and
variances of effects of interest via network-consistent permutation testing. We
then apply our technique to real data from experiments conducted on the
messaging product at LinkedIn. We find empirical support for our model, and
evidence that the standard method of analysis for A/B tests underestimates the
impact of new features in one-to-one messaging contexts.

25
The categories of this paper are: patt-sol
14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
15
The categories of this paper are: nucl-ex
26
The categories of this paper are: bayes-an
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom