Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adaptive Temporal-Difference Learning for Policy Evaluation with
  Per-State Uncertainty Estimates   We consider the core reinforcement-learning problem of on-policy value
function approximation from a batch of trajectory data, and focus on various
issues of Temporal Difference (TD) learning and Monte Carlo (MC) policy
evaluation. The two methods are known to achieve complementary bias-variance
trade-off properties, with TD tending to achieve lower variance but potentially
higher bias. In this paper, we argue that the larger bias of TD can be a result
of the amplification of local approximation errors. We address this by
proposing an algorithm that adaptively switches between TD and MC in each
state, thus mitigating the propagation of errors. Our method is based on
learned confidence intervals that detect biases of TD estimates. We demonstrate
in a variety of policy evaluation tasks that this simple adaptive algorithm
performs competitively with the best approach in hindsight, suggesting that
learned confidence intervals are a powerful technique for adapting policy
evaluation to use TD or MC returns in a data-driven way.

11
The categories of this paper are: hep-lat
24
The categories of this paper are: dg-ga
5
The categories of this paper are: gr-qc
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
14
The categories of this paper are: math-ph
25
The categories of this paper are: patt-sol
26
The categories of this paper are: bayes-an
18
The categories of this paper are: econ
15
The categories of this paper are: nucl-ex
4
The categories of this paper are: cond-mat
35
The categories of this paper are: mtrl-th
0
The categories of this paper are: hep-ph