Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement
  Learning   Training deep reinforcement learning agents on environments with multiple
levels / scenes / conditions from the same task, has become essential for many
applications aiming to achieve generalization and domain transfer from
simulation to the real world. While such a strategy is helpful with
generalization, the use of multiple scenes significantly increases the variance
of samples collected for policy gradient computations. Current methods continue
to view this collection of scenes as a single Markov Decision Process (MDP)
with a common value function; however, we argue that it is better to treat the
collection as a single environment with multiple underlying MDPs. To this end,
we propose a dynamic value estimation (DVE) technique for these multiple-MDP
environments, motivated by the clustering effect observed in the value function
distribution across different scenes. The resulting agent is able to learn a
more accurate and scene-specific value function estimate (and hence the
advantage function), leading to a lower sample variance. Our proposed approach
is simple to accommodate with several existing implementations (like PPO, A3C)
and results in consistent improvements for a range of ProcGen environments and
the AI2-THOR framework based visual navigation task.

3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
36
The categories of this paper are: cmp-lg
19
The categories of this paper are: eess
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
7
The categories of this paper are: hep-th
29
The categories of this paper are: funct-an
22
The categories of this paper are: chao-dyn
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
0
The categories of this paper are: hep-ph