Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Bias-reduced Multi-step Hindsight Experience Replay for Efficient
  Multi-goal Reinforcement Learning   Multi-goal reinforcement learning is widely applied in planning and robot
manipulation. Two main challenges in multi-goal reinforcement learning are
sparse rewards and sample inefficiency. Hindsight Experience Replay (HER) aims
to tackle the two challenges via goal relabeling. However, HER-related works
still need millions of samples and a huge computation. In this paper, we
propose Multi-step Hindsight Experience Replay (MHER), incorporating multi-step
relabeled returns based on $n$-step relabeling to improve sample efficiency.
Despite the advantages of $n$-step relabeling, we theoretically and
experimentally prove the off-policy $n$-step bias introduced by $n$-step
relabeling may lead to poor performance in many environments. To address the
above issue, two bias-reduced MHER algorithms, MHER($\lambda$) and Model-based
MHER (MMHER) are presented. MHER($\lambda$) exploits the $\lambda$ return while
MMHER benefits from model-based value expansions. Experimental results on
numerous multi-goal robotic tasks show that our solutions can successfully
alleviate off-policy $n$-step bias and achieve significantly higher sample
efficiency than HER and Curriculum-guided HER with little additional
computation beyond HER.

26
The categories of this paper are: bayes-an
17
The categories of this paper are: q-fin
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
2
The categories of this paper are: math
15
The categories of this paper are: nucl-ex
7
The categories of this paper are: hep-th
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas