Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Best Arm Identification in Restless Markov Multi-Armed Bandits   We study the problem of identifying the best arm in a multi-armed bandit
environment when each arm is a time-homogeneous and ergodic discrete-time
Markov process on a common, finite state space. The state evolution on each arm
is governed by the arm's transition probability matrix (TPM). A decision entity
that knows the set of arm TPMs but not the exact mapping of the TPMs to the
arms, wishes to find the index of the best arm as quickly as possible, subject
to an upper bound on the error probability. The decision entity selects one arm
at a time sequentially, and all the unselected arms continue to undergo state
evolution ({\em restless} arms). For this problem, we derive the first-known
problem instance-dependent asymptotic lower bound on the growth rate of the
expected time required to find the index of the best arm, where the asymptotics
is as the error probability vanishes. Further, we propose a sequential policy
that, for an input parameter $R$, forcibly selects an arm that has not been
selected for $R$ consecutive time instants. We show that this policy achieves
an upper bound that depends on $R$ and is monotonically non-increasing as
$R\to\infty$. The question of whether, in general, the limiting value of the
upper bound as $R\to\infty$ matches with the lower bound, remains open. We
identify a special case in which the upper and the lower bounds match. Prior
works on best arm identification have dealt with (a) independent and
identically distributed observations from the arms, and (b) rested Markov arms,
whereas our work deals with the more difficult setting of restless Markov arms.

23
The categories of this paper are: solv-int
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
3
The categories of this paper are: physics
15
The categories of this paper are: nucl-ex
14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
33
The categories of this paper are: plasm-ph
32
The categories of this paper are: atom-ph