Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An Evaluation Framework for Personalization Strategy Experiment Designs   Online Controlled Experiments (OCEs) are the gold standard in evaluating the
effectiveness of changes to websites. An important type of OCE evaluates
different personalization strategies, which present challenges in low test
power and lack of full control in group assignment. We argue that getting the
right experiment setup -- the allocation of users to treatment/analysis groups
-- should take precedence of post-hoc variance reduction techniques in order to
enable the scaling of the number of experiments. We present an evaluation
framework that, along with a few simple rule of thumbs, allow experimenters to
quickly compare which experiment setup will lead to the highest probability of
detecting a treatment effect under their particular circumstance.

17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
1
The categories of this paper are: cs
7
The categories of this paper are: hep-th
34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
27
The categories of this paper are: comp-gas
35
The categories of this paper are: mtrl-th
26
The categories of this paper are: bayes-an
18
The categories of this paper are: econ