Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unifying Gradient Estimators for Meta-Reinforcement Learning via
  Off-Policy Evaluation   Model-agnostic meta-reinforcement learning requires estimating the Hessian
matrix of value functions. This is challenging from an implementation
perspective, as repeatedly differentiating policy gradient estimates may lead
to biased Hessian estimates. In this work, we provide a unifying framework for
estimating higher-order derivatives of value functions, based on off-policy
evaluation. Our framework interprets a number of prior approaches as special
cases and elucidates the bias and variance trade-off of Hessian estimates. This
framework also opens the door to a new family of estimates, which can be easily
implemented with auto-differentiation libraries, and lead to performance gains
in practice.

23
The categories of this paper are: solv-int
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
24
The categories of this paper are: dg-ga
2
The categories of this paper are: math
27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
37
The categories of this paper are: supr-con
4
The categories of this paper are: cond-mat
6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess
21
The categories of this paper are: adap-org
13
The categories of this paper are: quant-ph
16
The categories of this paper are: stat
10
The categories of this paper are: q-bio
9
The categories of this paper are: nlin