Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Analysis of a Target-Based Actor-Critic Algorithm with Linear Function
  Approximation   Actor-critic methods integrating target networks have exhibited a stupendous
empirical success in deep reinforcement learning. However, a theoretical
understanding of the use of target networks in actor-critic methods is largely
missing in the literature. In this paper, we reduce this gap between theory and
practice by proposing the first theoretical analysis of an online target-based
actor-critic algorithm with linear function approximation in the discounted
reward setting. Our algorithm uses three different timescales: one for the
actor and two for the critic. Instead of using the standard single timescale
temporal difference (TD) learning algorithm as a critic, we use a two
timescales target-based version of TD learning closely inspired from practical
actor-critic algorithms implementing target networks. First, we establish
asymptotic convergence results for both the critic and the actor under
Markovian sampling. Then, we provide a finite-time analysis showing the impact
of incorporating a target network into actor-critic methods.

22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
18
The categories of this paper are: econ
28
The categories of this paper are: alg-geom
3
The categories of this paper are: physics
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
35
The categories of this paper are: mtrl-th
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
10
The categories of this paper are: q-bio
23
The categories of this paper are: solv-int