Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Bellman Residual Orthogonalization for Offline Reinforcement Learning   We introduce a new reinforcement learning principle that approximates the
Bellman equations by enforcing their validity only along an user-defined space
of test functions. Focusing on applications to model-free offline RL with
function approximation, we exploit this principle to derive confidence
intervals for off-policy evaluation, as well as to optimize over policies
within a prescribed policy class. We prove an oracle inequality on our policy
optimization procedure in terms of a trade-off between the value and
uncertainty of an arbitrary comparator policy. Different choices of test
function spaces allow us to tackle different problems within a common
framework. We characterize the loss of efficiency in moving from on-policy to
off-policy data using our procedures, and establish connections to
concentrability coefficients studied in past work. We examine in depth the
implementation of our methods with linear function approximation, and provide
theoretical guarantees with polynomial-time implementations even when Bellman
closure does not hold.

32
The categories of this paper are: atom-ph
21
The categories of this paper are: adap-org
17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
14
The categories of this paper are: math-ph
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
27
The categories of this paper are: comp-gas
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
29
The categories of this paper are: funct-an
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
22
The categories of this paper are: chao-dyn