  In this article, we report on the efficiency and effectiveness of multiagent
reinforcement learning methods (MARL) for the computation of flight delays to
resolve congestion problems in the Air Traffic Management (ATM) domain.
Specifically, we aim to resolve cases where demand of airspace use exceeds
capacity (demand-capacity problems), via imposing ground delays to flights at
the pre-tactical stage of operations (i.e. few days to few hours before
operation). Casting this into the multiagent domain, agents, representing
flights, need to decide on own delays w.r.t. own preferences, having no
information about others' payoffs, preferences and constraints, while they plan
to execute their trajectories jointly with others, adhering to operational
constraints. Specifically, we formalize the problem as a multiagent Markov
Decision Process (MA-MDP) and we show that it can be considered as a Markov
game in which interacting agents need to reach an equilibrium: What makes the
problem more interesting is the dynamic setting in which agents operate, which
is also due to the unforeseen, emergent effects of their decisions in the whole
system. We propose collaborative multiagent reinforcement learning methods to
resolve demand-capacity imbalances: Extensive experimental study on real-world
cases, shows the potential of the proposed approaches in resolving problems,
while advanced visualizations provide detailed views towards understanding the
quality of solutions provided.
