  WiFi densification leads to the existence of multiple overlapping coverage
areas, which allows user stations (STAs) to choose between different Access
Points (APs). The standard WiFi association method makes the STAs select the AP
with the strongest signal, which in many cases leads to underutilization of
some APs while overcrowding others. To mitigate this situation, Reinforcement
Learning techniques such as Multi-Armed Bandits can be used to dynamically
learn the optimal mapping between APs and STAs, and so redistribute the STAs
among the available APs accordingly. This is an especially challenging problem
since the network response observed by a given STA depends on the behavior of
the others, and so it is very difficult to predict without a global view of the
network.
  In this paper, we focus on solving this problem in a decentralized way, where
STAs independently explore the different APs inside their coverage range, and
select the one that better satisfy their needs. To do it, we propose a novel
approach called Opportunistic epsilon-greedy with Stickiness that halts the
exploration when a suitable AP is found, only resuming the exploration after
several unsatisfactory association rounds. With this approach, we reduce
significantly the network response dynamics, improving the ability of the STAs
to find a solution faster, as well as achieving a more efficient use of the
network resources.
  We investigate how the characteristics of the scenario (position of the APs
and STAs, traffic loads, and channel allocation strategies) impact the learning
process and the achievable performance. We also show that not all the STAs have
to implement the proposed solution to improve their performance. Finally, we
study the case where stations arrive progressively to the system, showing that
the considered approach is also suitable in such a non-stationary set-up.
