Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed
  Bandit with Many Arms   We study a Bayesian $k$-armed bandit problem in many-armed regime, when $k
\geq \sqrt{T}$, with $T$ the time horizon. We first show that subsampling is
critical for designing optimal policies. Specifically, the standard UCB
algorithm is sub-optimal while a subsampled UCB (SS-UCB), which samples
$\Theta(\sqrt{T})$ arms and executes UCB on that subset, is rate-optimal.
Despite theoretically optimal regret, SS-UCB numerically performs worse than a
greedy algorithm that pulls the current empirically best arm each time. These
empirical insights hold in a contextual setting as well, using simulations on
real data. These results suggest a new form of free exploration in the
many-armed regime that benefits greedy algorithms. We theoretically show that
this source of free exploration is deeply connected to the distribution of a
tail event for the prior distribution of arm rewards. This is a fundamentally
distinct phenomenon from free exploration due to variation in covariates, as
discussed in the recent literature on contextual bandits. Building on this
result, we prove that the subsampled greedy algorithm is rate-optimal for
Bernoulli bandits in many armed regime, and achieves sublinear regret with more
general distributions. Taken together, our results suggest that practitioners
may benefit from using greedy algorithms in the many-armed regime.

5
The categories of this paper are: gr-qc
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
12
The categories of this paper are: nucl-th
34
The categories of this paper are: chem-ph
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics