text: Managing interference in a network of macrocells underlaid with femtocells presents an important, yet challenging problem. A majority of spatial (frequency time) reuse based approaches partition the users based on coloring the interference graph, which is shown to be suboptimal. Some spatial time reuse based approaches schedule the maximal independent sets (MISs) in a cyclic, (weighted) round-robin fashion, which is inefficient for delay-sensitive applications. Our proposed policies schedule the MISs in a non-cyclic fashion, which aim to optimize any given network performance criterion for delay-sensitive applications while fulfilling minimum throughput requirements of the users. Importantly, we do not take the interference graph as given as in existing works; we propose an optimal construction of the interference graph. We prove that under certain conditions, the proposed policy achieves the optimal network performance. For large networks, we propose a low-complexity algorithm for computing the proposed policy. We show that the policy computed achieves a constant competitive ratio (with respect to the optimal network performance), which is independent of the network size, under wide range of deployment scenarios. The policy can be implemented in a decentralized manner by the users. Compared to the existing policies, our proposed policies can achieve improvement of up to 130 in large-scale deployments.
  @cite_14: In this paper, a decentralized and self-organizing mechanism for small cell networks (such as micro-, femto- and picocells) is proposed. In particular, an application to the case in which small cell networks aim to mitigate the interference caused to the macrocell network, while maximizing their own spectral efficiencies, is presented. The proposed mechanism is based on new notions of reinforcement learning (RL) through which small cells jointly estimate their time-average performance and optimize their probability distributions with which they judiciously choose their transmit configurations. Here, a minimum signal to interference plus noise ratio (SINR) is guaranteed at the macrocell user equipment (UE), while the small cells maximize their individual performances. The proposed RL procedure is fully distributed as every small cell base station requires only an observation of its instantaneous performance which can be obtained from its UE. Furthermore, it is shown that the proposed mechanism always converges to an epsilon Nash equilibrium when all small cells share the same interest. In addition, this mechanism is shown to possess better convergence properties and incur less overhead than existing techniques such as best response dynamics, fictitious play or classical RL. Finally, numerical results are given to validate the theoretical findings, highlighting the inherent tradeoffs facing small cells, namely exploration exploitation, myopic foresighted behavior and complete incomplete information.
  @cite_34: We introduce two mechanisms for interference mitigation, inspired by evolutionary game theory and machine learning to support the coexistence of a macrocell network underlaid with self-organized femtocell networks. In the first approach, stand-alone femtocells choose their strategies, observe the behavior of other players, and make the best decision based on their instantaneous payoff, as well as the average payoff of all other femtocells. We formulate the interactions among selfish femtocells using evolutionary games and demonstrate how the system converges to an equilibrium. In contrast, in the Reinforcement-Learning (RL) approach, information exchange among femtocells is no longer possible and hence each femtocell adapts its strategy and gradually learns by interacting with its environment (i.e., neighboring interferers) through trials-and-errors. Our investigations reveal that through learning, femtocells are able to self-organize by relying only on local information, while mitigating the interference towards the macrocell network. Besides, a trade-off exists where faster convergence is obtained in the evolutionary case as compared to the RL approach, at the expense of more side information. Finally, it is shown that femtocells face an interesting tradeoff of exploration versus exploitation in their learning processes.
summary: Besides the above two categories, there are several other related works. For instance in @cite_34 @cite_14 , the authors propose reinforcement learning and evolutionary learning techniques for the femtocells to learn efficient interference management policies. In @cite_34 , the femtocells learn the fixed transmit power levels, while in @cite_14 , the femtocells learn to randomize over different transmit power levels. However, the interference management policies in @cite_34 and @cite_14 cannot provide minimum throughput guarantees for the UEs. In constrast, we provide rigorous minimum throughput guarantees for the UEs. In both @cite_34 @cite_14 the femtocell UEs need to limit their transmission powers in every time slot such that the SINR of the macrocell UE is sufficiently high. If there is strong interference between some femtocells and the macrocell, the femtocell UEs will always transmit at lower power levels, leading to a low sum throughput for them.