  Learning-based methods, particularly Reinforcement Learning (RL), hold great
promise for streamlining deployment, enhancing performance, and achieving
generalization in the control of autonomous multirotor aerial vehicles. Deep RL
has been able to control complex systems with impressive fidelity and agility
in simulation but the simulation-to-reality transfer often brings a
hard-to-bridge reality gap. Moreover, RL is commonly plagued by prohibitively
long training times. In this work, we propose a novel asymmetric
actor-critic-based architecture coupled with a highly reliable RL-based
training paradigm for end-to-end quadrotor control. We show how curriculum
learning and a highly optimized simulator enhance sample complexity and lead to
fast training times. To precisely discuss the challenges related to
low-level/end-to-end multirotor control, we also introduce a taxonomy that
classifies the existing levels of control abstractions as well as
non-linearities and domain parameters. Our framework enables
Simulation-to-Reality (Sim2Real) transfer for direct RPM control after only 18
seconds of training on a consumer-grade laptop as well as its deployment on
microcontrollers to control a multirotor under real-time guarantees. Finally,
our solution exhibits competitive performance in trajectory tracking, as
demonstrated through various experimental comparisons with existing
state-of-the-art control solutions using a real Crazyflie nano quadrotor. We
open source the code including a very fast multirotor dynamics simulator that
can simulate about 5 months of flight per second on a laptop GPU. The fast
training times and deployment to a cheap, off-the-shelf quadrotor lower the
barriers to entry and help democratize the research and development of these
systems.
