Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Variance Penalized On-Policy and Off-Policy Actor-Critic   Reinforcement learning algorithms are typically geared towards optimizing the
expected return of an agent. However, in many practical applications, low
variance in the return is desired to ensure the reliability of an algorithm. In
this paper, we propose on-policy and off-policy actor-critic algorithms that
optimize a performance criterion involving both mean and variance in the
return. Previous work uses the second moment of return to estimate the variance
indirectly. Instead, we use a much simpler recently proposed direct variance
estimator which updates the estimates incrementally using temporal difference
methods. Using the variance-penalized criterion, we guarantee the convergence
of our algorithm to locally optimal policies for finite state action Markov
decision processes. We demonstrate the utility of our algorithm in tabular and
continuous MuJoCo domains. Our approach not only performs on par with
actor-critic and prior variance-penalization baselines in terms of expected
return, but also generates trajectories which have lower variance in the
return.

23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
32
The categories of this paper are: atom-ph
13
The categories of this paper are: quant-ph
6
The categories of this paper are: astro-ph
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
0
The categories of this paper are: hep-ph
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
24
The categories of this paper are: dg-ga
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
17
The categories of this paper are: q-fin
36
The categories of this paper are: cmp-lg