Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Expected Policy Gradients   We propose expected policy gradients (EPG), which unify stochastic policy
gradients (SPG) and deterministic policy gradients (DPG) for reinforcement
learning. Inspired by expected sarsa, EPG integrates across the action when
estimating the gradient, instead of relying only on the action in the sampled
trajectory. We establish a new general policy gradient theorem, of which the
stochastic and deterministic policy gradient theorems are special cases. We
also prove that EPG reduces the variance of the gradient estimates without
requiring deterministic policies and, for the Gaussian case, with no
computational overhead. Finally, we show that it is optimal in a certain sense
to explore with a Gaussian policy such that the covariance is proportional to
the exponential of the scaled Hessian of the critic with respect to the
actions. We present empirical results confirming that this new form of
exploration substantially outperforms DPG with the Ornstein-Uhlenbeck heuristic
in four challenging MuJoCo domains.

29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
9
The categories of this paper are: nlin
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
37
The categories of this paper are: supr-con
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
11
The categories of this paper are: hep-lat
33
The categories of this paper are: plasm-ph
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas