Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Infinite-Horizon Offline Reinforcement Learning with Linear Function
  Approximation: Curse of Dimensionality and Algorithm   In this paper, we investigate the sample complexity of policy evaluation in
infinite-horizon offline reinforcement learning (also known as the off-policy
evaluation problem) with linear function approximation. We identify a hard
regime $d\gamma^{2}>1$, where $d$ is the dimension of the feature vector and
$\gamma$ is the discount rate. In this regime, for any $q\in[\gamma^{2},1]$, we
can construct a hard instance such that the smallest eigenvalue of its feature
covariance matrix is $q/d$ and it requires
$\Omega\left(\frac{d}{\gamma^{2}\left(q-\gamma^{2}\right)\varepsilon^{2}}\exp\left(\Theta\left(d\gamma^{2}\right)\right)\right)$
samples to approximate the value function up to an additive error
$\varepsilon$. Note that the lower bound of the sample complexity is
exponential in $d$. If $q=\gamma^{2}$, even infinite data cannot suffice. Under
the low distribution shift assumption, we show that there is an algorithm that
needs at most $O\left(\max\left\{ \frac{\left\Vert \theta^{\pi}\right\Vert
_{2}^{4}}{\varepsilon^{4}}\log\frac{d}{\delta},\frac{1}{\varepsilon^{2}}\left(d+\log\frac{1}{\delta}\right)\right\}
\right)$ samples ($\theta^{\pi}$ is the parameter of the policy in linear
function approximation) and guarantees approximation to the value function up
to an additive error of $\varepsilon$ with probability at least $1-\delta$.

13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
0
The categories of this paper are: hep-ph
18
The categories of this paper are: econ
27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int