Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Some Limit Properties of Markov Chains Induced by Stochastic Recursive
  Algorithms   Recursive stochastic algorithms have gained significant attention in the
recent past due to data driven applications. Examples include stochastic
gradient descent for solving large-scale optimization problems and empirical
dynamic programming algorithms for solving Markov decision problems. These
recursive stochastic algorithms approximate certain contraction operators and
can be viewed within the framework of iterated random operators. Accordingly,
we consider iterated random operators over a Polish space that simulate
iterated contraction operator over that Polish space. Assume that the iterated
random operators are indexed by certain batch sizes such that as batch sizes
grow to infinity, each realization of the random operator converges (in some
sense) to the contraction operator it is simulating. We show that starting from
the same initial condition, the distribution of the random sequence generated
by the iterated random operators converges weakly to the trajectory generated
by the contraction operator. We further show that under certain conditions, the
time average of the random sequence converges to the spatial mean of the
invariant distribution. We then apply these results to logistic regression,
empirical value iteration, and empirical Q value iteration for finite state
finite action MDPs to illustrate the general theory develop here.

6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
3
The categories of this paper are: physics
36
The categories of this paper are: cmp-lg
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
9
The categories of this paper are: nlin
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
21
The categories of this paper are: adap-org
13
The categories of this paper are: quant-ph