Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Near-optimal Offline Reinforcement Learning with Linear Representation:
  Leveraging Variance Information with Pessimism   Offline reinforcement learning, which seeks to utilize offline/historical
data to optimize sequential decision-making strategies, has gained surging
prominence in recent studies. Due to the advantage that appropriate function
approximators can help mitigate the sample complexity burden in modern
reinforcement learning problems, existing endeavors usually enforce powerful
function representation models (e.g. neural networks) to learn the optimal
policies. However, a precise understanding of the statistical limits with
function representations, remains elusive, even when such a representation is
linear.
  Towards this goal, we study the statistical limits of offline reinforcement
learning with linear model representations. To derive the tight offline
learning bound, we design the variance-aware pessimistic value iteration
(VAPVI), which adopts the conditional variance information of the value
function for time-inhomogeneous episodic linear Markov decision processes
(MDPs). VAPVI leverages estimated variances of the value functions to reweight
the Bellman residuals in the least-square pessimistic value iteration and
provides improved offline learning bounds over the best-known existing results
(whereas the Bellman residuals are equally weighted by design). More
importantly, our learning bounds are expressed in terms of system quantities,
which provide natural instance-dependent characterizations that previous
results are short of. We hope our results draw a clearer picture of what
offline learning should look like when linear representations are provided.

18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
25
The categories of this paper are: patt-sol
2
The categories of this paper are: math
10
The categories of this paper are: q-bio
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
21
The categories of this paper are: adap-org
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex
32
The categories of this paper are: atom-ph
6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin