  We consider cooperative communications with energy harvesting (EH) relays,
and develop a distributed power control mechanism for the relaying terminals.
Unlike prior art which mainly deal with single-relay systems with saturated
traffic flow, we address the case of bursty data arrival at the source
cooperatively forwarded by multiple half-duplex EH relays. We aim at optimizing
the long-run average delay of the source packets under the energy neutrality
constraint on power consumption of each relay. While EH relay systems have been
predominantly optimized using either offline or online methodologies, we take
on a more realistic learning-theoretic approach. Hence, our scheme can be
deployed for real-time operation without assuming acausal information on
channel realizations, data/energy arrivals as required by offline optimization,
nor does it rely on precise statistics of the system processes as is the case
with online optimization. We formulate the problem as a partially observable
identical payoff stochastic game (PO-IPSG) with factored controllers, in which
the power control policy of each relay is adaptive to its local
source-to-relay/relay-to-destination channel states, its local energy state as
well as to the source buffer state information. We derive a multi-agent
reinforcement learning algorithm which is convergent to a locally optimal
solution of the formulated PO-IPSG. The proposed algorithm operates without
explicit message exchange between the relays, while inducing only little
source-relay signaling overhead. By simulation, we contrast the delay
performance of the proposed method against existing heuristics for throughput
maximization. It is shown that compared with these heuristics, the systematic
approach adopted in this paper has a smaller sub-optimality gap once evaluated
against a centralized optimal policy armed with perfect statistics.
