Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Work in Progress: Temporally Extended Auxiliary Tasks   Predictive auxiliary tasks have been shown to improve performance in numerous
reinforcement learning works, however, this effect is still not well
understood. The primary purpose of the work presented here is to investigate
the impact that an auxiliary task's prediction timescale has on the agent's
policy performance. We consider auxiliary tasks which learn to make on-policy
predictions using temporal difference learning. We test the impact of
prediction timescale using a specific form of auxiliary task in which the input
image is used as the prediction target, which we refer to as temporal
difference autoencoders (TD-AE). We empirically evaluate the effect of TD-AE on
the A2C algorithm in the VizDoom environment using different prediction
timescales. While we do not observe a clear relationship between the prediction
timescale on performance, we make the following observations: 1) using
auxiliary tasks allows us to reduce the trajectory length of the A2C algorithm,
2) in some cases temporally extended TD-AE performs better than a straight
autoencoder, 3) performance with auxiliary tasks is sensitive to the weight
placed on the auxiliary loss, 4) despite this sensitivity, auxiliary tasks
improved performance without extensive hyper-parameter tuning. Our overall
conclusions are that TD-AE increases the robustness of the A2C algorithm to the
trajectory length and while promising, further study is required to fully
understand the relationship between auxiliary task prediction timescale and the
agent's performance.

17
The categories of this paper are: q-fin
35
The categories of this paper are: mtrl-th
26
The categories of this paper are: bayes-an
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
29
The categories of this paper are: funct-an
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
3
The categories of this paper are: physics
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas
18
The categories of this paper are: econ
9
The categories of this paper are: nlin
32
The categories of this paper are: atom-ph
11
The categories of this paper are: hep-lat