Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Variance-reduced $Q$-learning is minimax optimal   We introduce and analyze a form of variance-reduced $Q$-learning. For
$\gamma$-discounted MDPs with finite state space $\mathcal{X}$ and action space
$\mathcal{U}$, we prove that it yields an $\epsilon$-accurate estimate of the
optimal $Q$-function in the $\ell_\infty$-norm using $\mathcal{O}
\left(\left(\frac{D}{ \epsilon^2 (1-\gamma)^3} \right) \; \log \left(
\frac{D}{(1-\gamma)} \right) \right)$ samples, where $D = |\mathcal{X}| \times
|\mathcal{U}|$. This guarantee matches known minimax lower bounds up to a
logarithmic factor in the discount complexity. In contrast, our past work shows
that ordinary $Q$-learning has worst-case quartic scaling in the discount
complexity.

23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
33
The categories of this paper are: plasm-ph
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph
20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
7
The categories of this paper are: hep-th