Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Information-Directed Exploration for Deep Reinforcement Learning   Efficient exploration remains a major challenge for reinforcement learning.
One reason is that the variability of the returns often depends on the current
state and action, and is therefore heteroscedastic. Classical exploration
strategies such as upper confidence bound algorithms and Thompson sampling fail
to appropriately account for heteroscedasticity, even in the bandit setting.
Motivated by recent findings that address this issue in bandits, we propose to
use Information-Directed Sampling (IDS) for exploration in reinforcement
learning. As our main contribution, we build on recent advances in
distributional reinforcement learning and propose a novel, tractable
approximation of IDS for deep Q-learning. The resulting exploration strategy
explicitly accounts for both parametric uncertainty and heteroscedastic
observation noise. We evaluate our method on Atari games and demonstrate a
significant improvement over alternative approaches.

7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
36
The categories of this paper are: cmp-lg
29
The categories of this paper are: funct-an
15
The categories of this paper are: nucl-ex
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
6
The categories of this paper are: astro-ph
26
The categories of this paper are: bayes-an
23
The categories of this paper are: solv-int
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
19
The categories of this paper are: eess
28
The categories of this paper are: alg-geom