  In this paper, we consider a wireless network consisting of a base station
(BS) that is serving multiple real-time traffic streams forwarding information
updates to their destinations in order to sustain the freshness of information
for time-critical applications. Since the wireless channels may be unreliable
due to the impurities of the propagation environments, such as deep fading,
blockages, etc., we integrate a reconfigurable intelligent surface (RIS) to the
wireless system in order to mitigate the propagation-induced impairments,
enhance the quality of the wireless links, and ensure that the required
freshness of information is achieved for these real-time applications. For this
network set-up, we investigate the joint optimization of the traffic streams
scheduling and the RIS phase-shift matrix with the goal of minimizing the
long-term average Age of Information (AoI). The formulated optimization problem
is a mixed-integer non-convex optimization problem, which is difficult to
solve. To circumvent the high-coupled optimization variables, and with the aid
of bilevel optimization, we decompose the original problem into an outer
traffic stream scheduling problem and an inner RIS phase-shift matrix problem.
For the outer problem, owing to its complexity and stochastic nature of packet
arrivals, we resort to deep reinforcement learning (DRL) solution where the
traffic stream scheduling is modeled as a Markov Decision Process (MDP), and
Proximal Policy Optimization (PPO) is invoked to solve it. Whereas, the inner
problem that determines the RIS configuration is solved through semi-definite
relaxation (SDR). Finally, we show through extensive simulations that our
approach evaluates the combined impact of scheduling policy and RIS
configuration on the long-term average AoI, where we demonstrate its
superiority against other baseline schemes.
