Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ClipUp: A Simple and Powerful Optimizer for Distribution-based Policy
  Evolution   Distribution-based search algorithms are an effective approach for
evolutionary reinforcement learning of neural network controllers. In these
algorithms, gradients of the total reward with respect to the policy parameters
are estimated using a population of solutions drawn from a search distribution,
and then used for policy optimization with stochastic gradient ascent. A common
choice in the community is to use the Adam optimization algorithm for obtaining
an adaptive behavior during gradient ascent, due to its success in a variety of
supervised learning settings. As an alternative to Adam, we propose to enhance
classical momentum-based gradient ascent with two simple techniques: gradient
normalization and update clipping. We argue that the resulting optimizer called
ClipUp (short for "clipped updates") is a better choice for distribution-based
policy evolution because its working principles are simple and easy to
understand and its hyperparameters can be tuned more intuitively in practice.
Moreover, it removes the need to re-tune hyperparameters if the reward scale
changes. Experiments show that ClipUp is competitive with Adam despite its
simplicity and is effective on challenging continuous control benchmarks,
including the Humanoid control task based on the Bullet physics simulator.

30
The categories of this paper are: q-alg
13
The categories of this paper are: quant-ph
36
The categories of this paper are: cmp-lg
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
5
The categories of this paper are: gr-qc
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph