Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: SoftDICE for Imitation Learning: Rethinking Off-policy Distribution
  Matching   We present SoftDICE, which achieves state-of-the-art performance for
imitation learning. SoftDICE fixes several key problems in ValueDICE, an
off-policy distribution matching approach for sample-efficient imitation
learning. Specifically, the objective of ValueDICE contains logarithms and
exponentials of expectations, for which the mini-batch gradient estimate is
always biased. Second, ValueDICE regularizes the objective with replay buffer
samples when expert demonstrations are limited in number, which however changes
the original distribution matching problem. Third, the re-parametrization trick
used to derive the off-policy objective relies on an implicit assumption that
rarely holds in training. We leverage a novel formulation of distribution
matching and consider an entropy-regularized off-policy objective, which yields
a completely offline algorithm called SoftDICE. Our empirical results show that
SoftDICE recovers the expert policy with only one demonstration trajectory and
no further on-policy/off-policy samples. SoftDICE also stably outperforms
ValueDICE and other baselines in terms of sample efficiency on Mujoco benchmark
tasks.

29
The categories of this paper are: funct-an
7
The categories of this paper are: hep-th
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
28
The categories of this paper are: alg-geom
25
The categories of this paper are: patt-sol
37
The categories of this paper are: supr-con
5
The categories of this paper are: gr-qc
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas