Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Indexability and Rollout Policy for Multi-State Partially Observable
  Restless Bandits   Restless multi-armed bandits with partially observable states has
applications in communication systems, age of information and recommendation
systems. In this paper, we study multi-state partially observable restless
bandit models. We consider three different models based on information
observable to decision maker -- 1) no information is observable from actions of
a bandit 2) perfect information from bandit is observable only for one action
on bandit, there is a fixed restart state, i.e., transition occurs from all
other states to that state 3) perfect state information is available to
decision maker for both actions on a bandit and there are two restart state for
two actions. We develop the structural properties. We also show a threshold
type policy and indexability for model 2 and 3. We present Monte Carlo (MC)
rollout policy. We use it for whittle index computation in case of model 2. We
obtain the concentration bound on value function in terms of horizon length and
number of trajectories for MC rollout policy. We derive explicit index formula
for model 3. We finally describe Monte Carlo rollout policy for model 1 when it
is difficult to show indexability. We demonstrate the numerical examples using
myopic policy, Monte Carlo rollout policy and Whittle index policy. We observe
that Monte Carlo rollout policy is good competitive policy to myopic.

15
The categories of this paper are: nucl-ex
24
The categories of this paper are: dg-ga
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
34
The categories of this paper are: chem-ph
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
32
The categories of this paper are: atom-ph
5
The categories of this paper are: gr-qc
12
The categories of this paper are: nucl-th