  We consider the power allocation of the physical layer and the buffer delay
of the upper application layer in energy harvesting green networks. The total
power required for reliable transmission includes the transmission power and
the circuit power. The harvested power (which is stored in a battery) and the
grid power constitute the power resource. The uncertainty of data generated
from the upper layer, the intermittence of the harvested energy, and the
variation of the fading channel are taken into account and described as
independent Markov processes. In each transmission, the transmitter decides the
transmission rate as well as the allocated power from the battery, and the rest
of the required power will be supplied by the power grid. The objective is to
find an allocation sequence of transmission rate and battery power to minimize
the long-term average buffer delay under the average grid power constraint. A
stochastic optimization problem is formulated accordingly to find such
transmission rate and battery power sequence. Furthermore, the optimization
problem is reformulated as a constrained MDP problem whose policy is a
two-dimensional vector with the transmission rate and the power allocation of
the battery as its elements. We prove that the optimal policy of the
constrained MDP can be obtained by solving the unconstrained MDP. Then we focus
on the analysis of the unconstrained average-cost MDP. The structural
properties of the average optimal policy are derived. Moreover, we discuss the
relations between elements of the two-dimensional policy. Next, based on the
theoretical analysis, the algorithm to find the constrained optimal policy is
presented for the finite state space scenario. In addition, heuristic policies
with low-complexity are given for the general state space. Finally, simulations
are performed under these policies to demonstrate the effectiveness.
