Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Convergence of Batch Asynchronous Stochastic Approximation With
  Applications to Reinforcement Learning   The stochastic approximation (SA) algorithm is a widely used probabilistic
method for finding a solution to an equation of the form
$\mathbf{f}(\boldsymbol{\theta}) = \mathbf{0}$ where $\mathbf{f} : \mathbb{R}^d
\rightarrow \mathbb{R}^d$, when only noisy measurements of $\mathbf{f}(\cdot)$
are available. In the literature to date, one can make a distinction between
"synchronous" updating, whereby the entire vector of the current guess
$\boldsymbol{\theta}_t$ is updated at each time, and "asynchronous" updating,
whereby ony one component of $\boldsymbol{\theta}_t$ is updated. In convex and
nonconvex optimization, there is also the notion of "batch" updating, whereby
some but not all components of $\boldsymbol{\theta}_t$ are updated at each time
$t$. In addition, there is also a distinction between using a "local" clock
versus a "global" clock. In the literature to date, convergence proofs when a
local clock is used make the assumption that the measurement noise is an i.i.d\
sequence, an assumption that does not hold in Reinforcement Learning (RL).
  In this note, we provide a general theory of convergence for batch
asymchronous stochastic approximation (BASA), that works whether the updates
use a local clock or a global clock, for the case where the measurement noises
form a martingale difference sequence. This is the most general result to date
and encompasses all others.

25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
22
The categories of this paper are: chao-dyn
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an