text: We develop a novel design framework for energy-efficient spectrum sharing among autonomous users who aim to minimize their energy consumptions subject to minimum throughput requirements. Most existing works proposed stationary spectrum sharing policies, in which users transmit at fixed power levels. Since users transmit simultaneously under stationary policies, to fulfill minimum throughput requirements, they need to transmit at high power levels to overcome interference. To improve energy efficiency, we construct nonstationary spectrum sharing policies, in which the users transmit at time-varying power levels. Specifically, we focus on TDMA (time-division multiple access) policies in which one user transmits at each time (but not in a round-robin fashion). The proposed policy can be implemented by each user running a low-complexity algorithm in a decentralized manner. It achieves high energy efficiency even when the users have erroneous and binary feedback about their interference levels. Moreover, it can adapt to the dynamic entry and exit of users. The proposed policy is also deviation-proof, namely autonomous users will find it in their self-interests to follow it. Compared to existing policies, the proposed policy can achieve an energy saving of up to 90 when the number of users is high.
  @cite_18: We propose decentralized cognitive MAC protocols that allow secondary users to independently search for spectrum opportunities without a central coordinator or a dedicated communication channel. Recognizing hardware and energy constraints, we assume that a secondary user may not be able to perform full-spectrum sensing or may not be willing to monitor the spectrum when it has no data to transmit. We develop an analytical framework for opportunistic spectrum access based on the theory of partially observable Markov decision process (POMDP). This decision-theoretic approach integrates the design of spectrum access protocols at the MAC layer with spectrum sensing at the physical layer and traffic statistics determined by the application layer of the primary network. It also allows easy incorporation of spectrum sensing error and constraint on the probability of colliding with the primary users. Under this POMDP framework, we propose cognitive MAC protocols that optimize the performance of secondary users while limiting the interference perceived by primary users. A suboptimal strategy with reduced complexity yet comparable performance is developed. Without additional control message exchange between the secondary transmitter and receiver, the proposed decentralized protocols ensure synchronous hopping in the spectrum between the transmitter and the receiver in the presence of collisions and spectrum sensing errors
  @cite_22: The authors study repeated games in which players observe a public outcome that imperfectly signals the actions played. They provide conditions guaranteeing that any feasible, individually rational payoff vector of the stage game can arise as a perfect equilibrium of the repeated game with sufficiently little discounting. The central condition requires that there exist action profiles with the property that, for any two players, no two deviations--one by either player--give rise to the same probability distribution over public outcomes. The results apply to principal-agent, partnership, oligopoly, and mechanism-design models, and to one-shot games with transferable utilities. Copyright 1994 by The Econometric Society.
  @cite_6: In communication systems where users share common resources, selfish behavior usually results in suboptimal resource utilization. There have been extensive works that model communication systems with selfish users as one-shot games and propose incentive schemes to achieve Pareto-optimal outcomes. However, in many communication systems, due to strong negative externalities among users, the sets of feasible payoffs in one-shot games are nonconvex. Thus, it is possible to expand the set of feasible payoffs by having users choose different action profiles in an alternating manner. In this paper, we formulate a model of repeated games with intervention. First, by using repeated games we can convexify the set of feasible payoffs in one-shot games. Second, by using intervention in repeated games we can achieve a larger set of equilibrium payoffs and loosen requirements for users' patience to achieve a target payoff. We study the problem of maximizing a welfare function defined on users' payoffs. We characterize the limit set of equilibrium payoffs. Given the optimal equilibrium payoff, we derive the sufficient condition on the discount factor and the intervention capability to achieve it, and design corresponding equilibrium strategies. We illustrate our analytical results with power control and flow control.
  @cite_19: We design distributed spectrum sensing and access strategies for opportunistic spectrum access (OSA) under an energy constraint on secondary users. Both the continuous and the bursty traffic models are considered for different applications of the secondary network. In each slot, a secondary user sequentially decides whether to sense, where in the spectrum to sense, and whether to access. By casting this sequential decision-making problem in the framework of partially observable Markov decision processes, we obtain stationary optimal spectrum sensing and access policies that maximize the throughput of the secondary user during its battery lifetime. We also establish threshold structures of the optimal policies and study the fundamental tradeoffs involved in the energy-constrained OSA design. Numerical results are provided to investigate the impact of the secondary user's residual energy on the optimal spectrum sensing and access decisions.
  @cite_16: We consider dynamic spectrum access where distributed secondary users search for spectrum opportunities without knowing the primary traffic statistics. In each slot, a secondary transmitter chooses one channel to sense and subsequently transmit if the channel is sensed as idle. Sensing is imperfect, i.e., an idle channel may be sensed as busy and vice versa. Without centralized control, each secondary user needs to independently identify the channels that offer the most opportunities while avoiding collisions with both primary and other secondary users. We address the problem within a cooperative game framework, where the objective is to maximize the throughput of the secondary network under a constraint on the collision with the primary system. The performance of a decentralized channel access policy is measured by the system regret, defined as the expected total performance loss with respect to the optimal performance in the ideal scenario where the traffic load of the primary system on each channel is known to all secondary users and collisions among secondary users are eliminated through centralized scheduling. By exploring the rich communication structure of the problem, we show that the optimal system regret has the same logarithmic order as in the centralized counterpart with perfect sensing. A decentralized policy is constructed to achieve the logarithmic order of the system regret. In a broader context, this work addresses imperfect reward observation in decentralized multi-armed bandit problems.
  @cite_17: We formulate and study a decentralized multi-armed bandit (MAB) problem. There are M distributed players competing for N independent arms. Each arm, when played, offers i.i.d. reward according to a distribution with an unknown parameter. At each time, each player chooses one arm to play without exchanging observations or any information with other players. Players choosing the same arm collide, and, depending on the collision model, either no one receives reward or the colliding players share the reward in an arbitrary way. We show that the minimum system regret of the decentralized MAB grows with time at the same logarithmic order as in the centralized counterpart where players act collectively as a single entity by exchanging observations and making decisions jointly. A decentralized policy is constructed to achieve this optimal order while ensuring fairness among players and without assuming any pre-agreement or information exchange among players. Based on a time-division fair sharing (TDFS) of the M best arms, the proposed policy is constructed and its order optimality is proven under a general reward model. Furthermore, the basic structure of the TDFS policy can be used with any order-optimal single-player policy to achieve order optimality in the decentralized setting. We also establish a lower bound on the system regret for a general class of decentralized polices, to which the proposed policy belongs. This problem finds potential applications in cognitive radio networks, multi-channel communication systems, multi-agent systems, web search and advertising, and social networks.
summary: There have been some works that develop nonstationary policies using repeated games @cite_6 @cite_22 , Markov decision processes (MDPs) @cite_18 @cite_19 , and multi-art bandit @cite_17 -- @cite_16 . We summarize the major differences between the existing nonstationary policies and our proposed policy in Table .