  This paper investigates the problem of age of information (AoI) aware radio
resource management for a platooning system. Multiple autonomous platoons
exploit the cellular wireless vehicle-to-everything (C-V2X) communication
technology to disseminate the cooperative awareness messages (CAMs) to their
followers while ensuring timely delivery of safety-critical messages to the
Road-Side Unit (RSU). Due to the challenges of dynamic channel conditions,
centralized resource management schemes that require global information are
inefficient and lead to large signaling overheads. Hence, we exploit a
distributed resource allocation framework based on multi-agent reinforcement
learning (MARL), where each platoon leader (PL) acts as an agent and interacts
with the environment to learn its optimal policy. Existing MARL algorithms
consider a holistic reward function for the group's collective success, which
often ends up with unsatisfactory results and cannot guarantee an optimal
policy for each agent. Consequently, motivated by the existing literature in
RL, we propose a novel MARL framework that trains two critics with the
following goals: A global critic which estimates the global expected reward and
motivates the agents toward a cooperating behavior and an exclusive local
critic for each agent that estimates the local individual reward. Furthermore,
based on the tasks each agent has to accomplish, the individual reward of each
agent is decomposed into multiple sub-reward functions where task-wise value
functions are learned separately. Numerical results indicate our proposed
algorithm's effectiveness compared with the conventional RL methods applied in
this area.
