Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adaptive Lambda Least-Squares Temporal Difference Learning   Temporal Difference learning or TD($\lambda$) is a fundamental algorithm in
the field of reinforcement learning. However, setting TD's $\lambda$ parameter,
which controls the timescale of TD updates, is generally left up to the
practitioner. We formalize the $\lambda$ selection problem as a bias-variance
trade-off where the solution is the value of $\lambda$ that leads to the
smallest Mean Squared Value Error (MSVE). To solve this trade-off we suggest
applying Leave-One-Trajectory-Out Cross-Validation (LOTO-CV) to search the
space of $\lambda$ values. Unfortunately, this approach is too computationally
expensive for most practical applications. For Least Squares TD (LSTD) we show
that LOTO-CV can be implemented efficiently to automatically tune $\lambda$ and
apply function optimization methods to efficiently search the space of
$\lambda$ values. The resulting algorithm, ALLSTD, is parameter free and our
experiments demonstrate that ALLSTD is significantly computationally faster
than the na\"{i}ve LOTO-CV implementation while achieving similar performance.

21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
32
The categories of this paper are: atom-ph
33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con
12
The categories of this paper are: nucl-th
34
The categories of this paper are: chem-ph
20
The categories of this paper are: acc-phys
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
28
The categories of this paper are: alg-geom
7
The categories of this paper are: hep-th
0
The categories of this paper are: hep-ph