  Herein, minimization of time-averaged age-of-information (AoI) in an energy
harvesting (EH) source equipped remote sensing setting is considered. The EH
source opportunistically samples one or multiple processes over discrete time
instants, and sends the status updates to a sink node over a time-varying
wireless link. At any discrete time instant, the EH node decides whether to
ascertain the link quality using its stored energy (called probing) and then,
decides whether to sample a process and communicate the data based on the
channel probe outcome. The trade-off is between the freshness of information
available at the sink node and the available energy at the energy buffer of the
source node. To this end, infinite horizon Markov decision process (MDP) theory
is used to formulate the problem of minimization of time-averaged expected AoI
for a single energy harvesting source node. The following two scenarios are
considered: (i) energy arrival process and channel fading process are
independent and identically distributed (i.i.d.) across time, (ii) energy
arrival process and channel fading process are Markovian. In the i.i.d.
setting, after probing a channel, the optimal source node sampling policy is
shown to be a threshold policy involving the instantaneous age of the process,
the available energy in the buffer and the instantaneous channel quality as the
decision variables. Also, for unknown channel state and energy harvesting
characteristics, a variant of the Q-learning algorithm is proposed for the
two-stage action taken by the source, that seeks to learn the optimal status
update policy over time. For Markovian channel and Markovian energy arrival
processes, the problem is again formulated as an MDP, and a learning algorithm
is provided to handle unknown dynamics. Finally, numerical results are provided
to demonstrate the policy structures and performance trade-offs.
