Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Is Temporal Difference Learning Optimal? An Instance-Dependent Analysis   We address the problem of policy evaluation in discounted Markov decision
processes, and provide instance-dependent guarantees on the $\ell_\infty$-error
under a generative model. We establish both asymptotic and non-asymptotic
versions of local minimax lower bounds for policy evaluation, thereby providing
an instance-dependent baseline by which to compare algorithms. Theory-inspired
simulations show that the widely-used temporal difference (TD) algorithm is
strictly suboptimal when evaluated in a non-asymptotic setting, even when
combined with Polyak-Ruppert iterate averaging. We remedy this issue by
introducing and analyzing variance-reduced forms of stochastic approximation,
showing that they achieve non-asymptotic, instance-dependent optimality up to
logarithmic factors.

3
The categories of this paper are: physics
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
12
The categories of this paper are: nucl-th
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
35
The categories of this paper are: mtrl-th
34
The categories of this paper are: chem-ph