Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible
  Off-Policy Evaluation   Off-policy evaluation (OPE) aims to estimate the performance of hypothetical
policies using data generated by a different policy. Because of its huge
potential impact in practice, there has been growing research interest in this
field. There is, however, no real-world public dataset that enables the
evaluation of OPE, making its experimental studies unrealistic and
irreproducible. With the goal of enabling realistic and reproducible OPE
research, we present Open Bandit Dataset, a public logged bandit dataset
collected on a large-scale fashion e-commerce platform, ZOZOTOWN. Our dataset
is unique in that it contains a set of multiple logged bandit datasets
collected by running different policies on the same platform. This enables
experimental comparisons of different OPE estimators for the first time. We
also develop Python software called Open Bandit Pipeline to streamline and
standardize the implementation of batch bandit algorithms and OPE. Our open
data and software will contribute to fair and transparent OPE research and help
the community identify fruitful research directions. We provide extensive
benchmark experiments of existing OPE estimators using our dataset and
software. The results open up essential challenges and new avenues for future
OPE research.

9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci