Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning   Exploration in complex domains is a key challenge in reinforcement learning,
especially for tasks with very sparse rewards. Recent successes in deep
reinforcement learning have been achieved mostly using simple heuristic
exploration strategies such as $\epsilon$-greedy action selection or Gaussian
control noise, but there are many tasks where these methods are insufficient to
make any learning progress. Here, we consider more complex heuristics:
efficient and scalable exploration strategies that maximize a notion of an
agent's surprise about its experiences via intrinsic motivation. We propose to
learn a model of the MDP transition probabilities concurrently with the policy,
and to form intrinsic rewards that approximate the KL-divergence of the true
transition probabilities from the learned model. One of our approximations
results in using surprisal as intrinsic motivation, while the other gives the
$k$-step learning progress. We show that our incentives enable agents to
succeed in a wide range of environments with high-dimensional state spaces and
very sparse rewards, including continuous control tasks and games in the Atari
RAM domain, outperforming several other heuristic exploration techniques.

36
The categories of this paper are: cmp-lg
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
23
The categories of this paper are: solv-int
16
The categories of this paper are: stat
34
The categories of this paper are: chem-ph
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
25
The categories of this paper are: patt-sol
18
The categories of this paper are: econ
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
22
The categories of this paper are: chao-dyn
4
The categories of this paper are: cond-mat
26
The categories of this paper are: bayes-an