Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Stateful Offline Contextual Policy Evaluation and Learning   We study off-policy evaluation and learning from sequential data in a
structured class of Markov decision processes that arise from repeated
interactions with an exogenous sequence of arrivals with contexts, which
generate unknown individual-level responses to agent actions. This model can be
thought of as an offline generalization of contextual bandits with resource
constraints. We formalize the relevant causal structure of problems such as
dynamic personalized pricing and other operations management problems in the
presence of potentially high-dimensional user types. The key insight is that an
individual-level response is often not causally affected by the state variable
and can therefore easily be generalized across timesteps and states. When this
is true, we study implications for (doubly robust) off-policy evaluation and
learning by instead leveraging single time-step evaluation, estimating the
expectation over a single arrival via data from a population, for fitted-value
iteration in a marginal MDP. We study sample complexity and analyze error
amplification that leads to the persistence, rather than attenuation, of
confounding error over time. In simulations of dynamic and capacitated pricing,
we show improved out-of-sample policy performance in this class of relevant
problems.

4
The categories of this paper are: cond-mat
35
The categories of this paper are: mtrl-th
33
The categories of this paper are: plasm-ph
13
The categories of this paper are: quant-ph
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
21
The categories of this paper are: adap-org