Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Finite-Sample Analysis of Decentralized Temporal-Difference Learning
  with Linear Function Approximation   Motivated by the emerging use of multi-agent reinforcement learning (MARL) in
engineering applications such as networked robotics, swarming drones, and
sensor networks, we investigate the policy evaluation problem in a fully
decentralized setting, using temporal-difference (TD) learning with linear
function approximation to handle large state spaces in practice. The goal of a
group of agents is to collaboratively learn the value function of a given
policy from locally private rewards observed in a shared environment, through
exchanging local estimates with neighbors. Despite their simplicity and
widespread use, our theoretical understanding of such decentralized TD learning
algorithms remains limited. Existing results were obtained based on i.i.d. data
samples, or by imposing an `additional' projection step to control the
`gradient' bias incurred by the Markovian observations. In this paper, we
provide a finite-sample analysis of the fully decentralized TD(0) learning
under both i.i.d. as well as Markovian samples, and prove that all local
estimates converge linearly to a small neighborhood of the optimum. The
resultant error bounds are the first of its type---in the sense that they hold
under the most practical assumptions ---which is made possible by means of a
novel multi-step Lyapunov analysis.

17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
4
The categories of this paper are: cond-mat
32
The categories of this paper are: atom-ph
18
The categories of this paper are: econ
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom