Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Distributed Value Function Approximation for Collaborative Multi-Agent
  Reinforcement Learning   In this paper we propose several novel distributed gradient-based temporal
difference algorithms for multi-agent off-policy learning of linear
approximation of the value function in Markov decision processes with strict
information structure constraints, limiting inter-agent communications to small
neighborhoods. The algorithms are composed of: 1) local parameter updates based
on single-agent off-policy gradient temporal difference learning algorithms,
including eligibility traces with state dependent parameters, and 2) linear
stochastic time varying consensus schemes, represented by directed graphs. The
proposed algorithms differ by their form, definition of eligibility traces,
selection of time scales and the way of incorporating consensus iterations. The
main contribution of the paper is a convergence analysis based on the general
properties of the underlying Feller-Markov processes and the stochastic time
varying consensus model. We prove, under general assumptions, that the
parameter estimates generated by all the proposed algorithms weakly converge to
the corresponding ordinary differential equations (ODE) with precisely defined
invariant sets. It is demonstrated how the adopted methodology can be applied
to temporal-difference algorithms under weaker information structure
constraints. The variance reduction effect of the proposed algorithms is
demonstrated by formulating and analyzing an asymptotic stochastic differential
equation. Specific guidelines for communication network design are provided.
The algorithms' superior properties are illustrated by characteristic
simulation results.

11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
28
The categories of this paper are: alg-geom
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
9
The categories of this paper are: nlin
29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
10
The categories of this paper are: q-bio
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
15
The categories of this paper are: nucl-ex