  As an emerging technology, Connected Autonomous Vehicles (CAVs) are believed
to have the ability to move through intersections in a faster and safer manner,
through effective Vehicle-to-Everything (V2X) communication and global
observation. Autonomous intersection management is a key path to efficient
crossing at intersections, which reduces unnecessary slowdowns and stops
through adaptive decision process of each CAV, enabling fuller utilization of
the intersection space. Distributed reinforcement learning (DRL) offers a
flexible, end-to-end model for AIM, adapting for many intersection scenarios.
While DRL is prone to collisions as the actions of multiple sides in the
complicated interactions are sampled from a generic policy, restricting the
application of DRL in realistic scenario. To address this, we propose a
hierarchical RL framework where models at different levels vary in receptive
scope, action step length, and feedback period of reward. The upper layer model
accelerate CAVs to prevent them from being clashed, while the lower layer model
adjust the trends from upper layer model to avoid the change of mobile state
causing new conflicts. And the real action of CAV at each step is co-determined
by the trends from both levels, forming a real-time balance in the adversarial
process. The proposed model is proven effective in the experiment undertaken in
a complicated intersection with 4 branches and 4 lanes each branch, and show
better performance compared with baselines.
