Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Optimal Policies for Observing Time Series and Related Restless Bandit
  Problems   The trade-off between the cost of acquiring and processing data, and
uncertainty due to a lack of data is fundamental in machine learning. A basic
instance of this trade-off is the problem of deciding when to make noisy and
costly observations of a discrete-time Gaussian random walk, so as to minimise
the posterior variance plus observation costs. We present the first proof that
a simple policy, which observes when the posterior variance exceeds a
threshold, is optimal for this problem. The proof generalises to a wide range
of cost functions other than the posterior variance.
  This result implies that optimal policies for linear-quadratic-Gaussian
control with costly observations have a threshold structure. It also implies
that the restless bandit problem of observing multiple such time series, has a
well-defined Whittle index. We discuss computation of that index, give
closed-form formulae for it, and compare the performance of the associated
index policy with heuristic policies.
  The proof is based on a new verification theorem that demonstrates threshold
structure for Markov decision processes, and on the relation between binary
sequences known as mechanical words and the dynamics of discontinuous nonlinear
maps, which frequently arise in physics, control and biology.

18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
2
The categories of this paper are: math
12
The categories of this paper are: nucl-th
14
The categories of this paper are: math-ph
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
5
The categories of this paper are: gr-qc
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
1
The categories of this paper are: cs
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
34
The categories of this paper are: chem-ph
20
The categories of this paper are: acc-phys