text: This paper studies online transmission policies for an energy harvesting transmitter. Unlike the existing online policies which more or less require the knowledge on the future behavior of the energy and data arrival processes, we consider a practical but significantly more challenging scenario where the energy and data arrival processes are assumed to be totally unknown. Our design is formulated as a robust-optimal control problem which aims to optimize the worst-case performance. The transmission policy is designed only based on the current battery energy level and the data queue length directly monitored by the transmitter itself. Specifically, we apply an event-trigger approach in which the transmitter continuously monitors the battery energy and data queue length, and triggers an event when a significant change occurs in either of them. Once an event is triggered, the transmission policy is updated by the solution to the robust-optimal control problem. We consider both the transmission time and throughput as the performance metrics and formulated two optimization problems. The solutions are given in either a simple analytical form or an easy-to-implement algorithm.
  @cite_20: In this paper, we show how to achieve close-to-optimal utility performance in energy-harvesting networks with only finite capacity energy storage devices. In these networks, nodes are capable of harvesting energy from the environment. The amount of energy that can be harvested is time-varying and evolves according to some probability law. We develop an online algorithm, called the Energy-limited Scheduling Algorithm (ESA), which jointly manages the energy and makes power allocation decisions for packet transmissions. ESA only has to keep track of the amount of energy left at the network nodes and does not require any knowledge of the harvestable energy process. We show that ESA achieves a utility that is within O(e) of the optimal, for any e > 0, while ensuring that the network congestion and the required capacity of the energy storage devices are deterministically upper-bounded by bounds of size O(1 e). We then also develop the Modified-ESA (MESA) algorithm to achieve the same O(e) close-to-utility performance, with the average network congestion and the required capacity of the energy storage devices being only O([log(1 e)]2), which is close to the theoretical lower bound O(log(1 e)).
  @cite_6: A point-to-point wireless communication system in which the transmitter is equipped with an energy harvesting device and a rechargeable battery, is studied. Both the energy and the data arrivals at the transmitter are modeled as Markov processes. Delay-limited communication is considered assuming that the underlying channel is block fading with memory, and the instantaneous channel state information is available at both the transmitter and the receiver. The expected total transmitted data during the transmitter's activation time is maximized under three different sets of assumptions regarding the information available at the transmitter about the underlying stochastic processes. A learning theoretic approach is introduced, which does not assume any a priori information on the Markov processes governing the communication system. In addition, online and offline optimization problems are studied for the same setting. Full statistical knowledge and causal information on the realizations of the underlying stochastic processes are assumed in the online optimization problem, while the offline optimization problem assumes non-causal knowledge of the realizations in advance. Comparing the optimal solutions in all three frameworks, the performance loss due to the lack of the transmitter's information regarding the behaviors of the underlying Markov processes is quantified.
summary: The second framework uses parameter-independent methodologies. @cite_6 , the energy and data arrival processes were formulated as time-homogeneous Markov chains without knowing the transition matrix, and Q-learning was applied to perform online optimization on the transmission policy. By Lyapunov optimization technique combined with the idea of weight perturbation, @cite_20 proposed a generic utility-maximization policy, under the assumption that the amount of harvested energy in each time slot is independent and identically distributed (i.i.d.) but its statistical parameters are totally unknown. Although these parameter-independent policies require less knowledge on the energy and data arrival processes, the stochastic models of the energy and data arrival processes still need to be known exactly.