  Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless
connectivity to static and mobile ground users in situations of increased
network demand or points of failure in existing terrestrial cellular
infrastructure. However, UAVs are energy-constrained and experience the
challenge of interference from nearby UAV cells sharing the same frequency
spectrum, thereby impacting the system's energy efficiency (EE). Recent
approaches focus on optimising the system's EE by optimising the trajectory of
UAVs serving only static ground users and neglecting mobile users. Several
others neglect the impact of interference from nearby UAV cells, assuming an
interference-free network environment. Despite growing research interest in
decentralised control over centralised UAVs' control, direct collaboration
among UAVs to improve coordination while optimising the systems' EE has not
been adequately explored. To address this, we propose a direct collaborative
communication-enabled multi-agent decentralised double deep Q-network
(CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows
UAVs to explicitly share their telemetry via existing 3GPP guidelines by
communicating with their nearest neighbours. This allows the agent-controlled
UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and
converging to optimal policies. Simulation results show that the proposed
approach outperforms existing baselines in terms of maximising the systems' EE
without degrading coverage performance in the network. The CMAD-DDQN approach
outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the
multi-agent deep deterministic policy gradient (MADDPG) and random policy
approaches that consider a 2D UAV deployment design while neglecting
interference from nearby UAV cells by about 15%, 65% and 85%, respectively.
