  In this paper, we investigate cost-aware joint learning and optimization for
multi-channel opportunistic spectrum access in a cognitive radio system. We
investigate a discrete time model where the time axis is partitioned into
frames. Each frame consists of a sensing phase, followed by a transmission
phase. During the sensing phase, the user is able to sense a subset of channels
sequentially before it decides to use one of them in the following transmission
phase. We assume the channel states alternate between busy and idle according
to independent Bernoulli random processes from frame to frame. To capture the
inherent uncertainty in channel sensing, we assume the reward of each
transmission when the channel is idle is a random variable. We also associate
random costs with sensing and transmission actions. Our objective is to
understand how the costs and reward of the actions would affect the optimal
behavior of the user in both offline and online settings, and design the
corresponding opportunistic spectrum access strategies to maximize the expected
cumulative net reward (i.e., reward-minus-cost). We start with an offline
setting where the statistics of the channel status, costs and reward are known
beforehand. We show that the the optimal policy exhibits a recursive double
threshold structure, and the user needs to compare the channel statistics with
those thresholds sequentially in order to decide its actions. With such
insights, we then study the online setting, where the statistical information
of the channels, costs and reward are unknown a priori. We judiciously balance
exploration and exploitation, and show that the cumulative regret scales in
O(log T). We also establish a matched lower bound, which implies that our
online algorithm is order-optimal. Simulation results corroborate our
theoretical analysis.
