Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An Emphatic Approach to the Problem of Off-policy Temporal-Difference
  Learning   In this paper we introduce the idea of improving the performance of
parametric temporal-difference (TD) learning algorithms by selectively
emphasizing or de-emphasizing their updates on different time steps. In
particular, we show that varying the emphasis of linear TD($\lambda$)'s updates
in a particular way causes its expected update to become stable under
off-policy training. The only prior model-free TD methods to achieve this with
per-step computation linear in the number of function approximation parameters
are the gradient-TD family of methods including TDC, GTD($\lambda$), and
GQ($\lambda$). Compared to these methods, our _emphatic TD($\lambda$)_ is
simpler and easier to use; it has only one learned parameter vector and one
step-size parameter. Our treatment includes general state-dependent discounting
and bootstrapping functions, and a way of specifying varying degrees of
interest in accurately valuing different states.

31
The categories of this paper are: ao-sci
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
15
The categories of this paper are: nucl-ex
26
The categories of this paper are: bayes-an
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph
2
The categories of this paper are: math
8
The categories of this paper are: hep-ex
19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
10
The categories of this paper are: q-bio
0
The categories of this paper are: hep-ph
27
The categories of this paper are: comp-gas