Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adaptive Approximate Policy Iteration   Model-free reinforcement learning algorithms combined with value function
approximation have recently achieved impressive performance in a variety of
application domains. However, the theoretical understanding of such algorithms
is limited, and existing results are largely focused on episodic or discounted
Markov decision processes (MDPs). In this work, we present adaptive approximate
policy iteration (AAPI), a learning scheme which enjoys a $\tilde{O}(T^{2/3})$
regret bound for undiscounted, continuing learning in uniformly ergodic MDPs.
This is an improvement over the best existing bound of $\tilde{O}(T^{3/4})$ for
the average-reward case with function approximation. Our algorithm and analysis
rely on online learning techniques, where value functions are treated as
losses. The main technical novelty is the use of a data-dependent adaptive
learning rate coupled with a so-called optimistic prediction of upcoming
losses. In addition to theoretical guarantees, we demonstrate the advantages of
our approach empirically on several environments.

28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics
2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
19
The categories of this paper are: eess
18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
8
The categories of this paper are: hep-ex
24
The categories of this paper are: dg-ga
36
The categories of this paper are: cmp-lg
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th